alternate types of rail

[voss749]

Cog rail for mountains

Light rail/trams including amsterdam/berlin trams, and Sbahn types

also rural trains including Kiha 100 series https://en.wikipedia.org/wiki/KiHa_100_series

[kamnet]

Any type of rail outside of the defaults can already be created via NewGRF. AFAIK there isn't currently a cog or rack & pinion rail NewGRF, but it's been discussed in the past.

Trams already are coded as an alternative to road vehicles, and are activated when you install a NewGRF tram set. You'll find trams, light rails, U-bahn and S-bahn vehicles in quite a few NewGRF sets. Check out 2CC Tram Set and 2CC Trains in NML. And I am pretty sure the KiHA 100 is encoded in the Japan Train Set.

[voss749]

the other one would be cable cars.

[kamnet]

the other one would be cable cars.

Yep, we got that one too: http://www.tt-forums.net/viewtopic.php?t=61471

You might be interested in exploring the in-game content download system, or looking at the OpenTTD wiki article NewGRF List to see a wide selection of what's available for download.