The paper proposes constitutive relations for the yield surface and dilatancy factor. The structural parameter in the relations is porosity whose variation under deformation changes the parameters describing the yield surface and dilatancy factor. The proposed relations allow one to describe the deformation of rocks in dilatancy and compaction modes as well as their alternation during the process and transition to slip with no dilatancy. On the example of a rock layer, the formation of strain localization or so-called shear bands is considered. It is shown that strain-induced variation in the properties of material can change its deformation modes such that dilatancy and compaction in adjacent regions will develop at a time. As a result, the deformation will proceed with no change of average stress in certain shear band regions
