This paper presents a corpus-based analysis of morphosyntactic errors made by Finnish Russian heritage speakers belonging to the second generation of Russian emigrants in Finland. This study focuses on the Russian prepositional phrase government, which is reportedly an unstable syntactic phenomenon in Russian varieties across the world. The research literature reports two regular contact-induced tendencies concerning violation of Russian grammatical rules after prepositions: a) a loss of case government and an overuse of the Nominative, which is a common non-standard phenomenon among foreign language learners of Russian b) a mixture of oblique case forms, which is typical for Russian dialects and spoken language. This study identifies the regular changes, detected in a group of Russian heritage speakers in Finland, and explores to what extent the non-standard government originates from the structural properties of the dominant Finnish language. The data covers written production of university students, which are high-proficient Russian-Finnish bilinguals. The corpus-based approach combines basic statistical metrics and linguistic analysis of non-standard prepositional phrases. The results indicate strong tendency to overuse the Genitive instead of the Prepositional and the Instrumental, meanwhile the Nominative, manifesting the loss of government, occurs rarely in the prepositional phrase. The study reveals that the speech of bilingual speakers of two inflecting languages with ramified systems of case marking practically does not develop analyticity and attrition of case government in prepositional phrases. The incomplete acquisition of government manifests itself as a mixture of correct and incorrect forms of oblique cases induced by the prepositional pattern of the dominant language.