The main cause: some titles are available only in pan and scan because there was no letterbox or anamorphic transfer made from film.
Since transfers cost $50,000 to $100,000, studios may not think a new transfer is justified. The original film or rights to it are no longer available for a new transfer. In the case of old movies, they were shot full frame in the 1.37 "academy" aspect ratio so no widescreen version can be created. Video shot with TV cameras (music concerts) is already in 4:3 format.