We propose a new pressure-based structure-preserving (SP) and quasi asymptotic preserving (AP) staggered semi-implicit finite volume scheme for the unified first order hyperbolic formulation of continuum mechanics [1], which goes back to the pioneering work of Godunov [2] and further work of Godunov #38; Romenski [3] and Peshkov #38; Romenski [4]. The unified model is based on the theory of symmetric-hyperbolic and thermodynamically compatible (SHTC) systems [2,5] and includes the description of elastic and elasto-plastic solids in the nonlinear large-strain regime as well as viscous and inviscid heat-conducting fluids, which correspond to the stiff relaxation limit of the model. In the absence of relaxation source terms, the homogeneous PDE system is endowed with two stationary linear differential constraints (involutions), which require the curl of distortion field and the curl of the thermal impulse to be zero for all times. In the stiff relaxation limit, the unified model tends asymptotically to the compressible Navier-Stokes equations. The new structure-preserving scheme presented in this paper can be proven to be exactly curl-free for the homogeneous part of the PDE system, i.e. in the absence of relaxation source terms. We furthermore prove that the scheme is quasi asymptotic preserving in the stiff relaxation limit, in the sense that the numerical scheme reduces to a consistent second order accurate discretization of the compressible Navier-Stokes equations when the relaxation times tend to zero. Last but not least, the proposed scheme is suitable for the simulation of all Mach number flows thanks to its conservative formulation and the implicit discretization of the pressure terms.