IPX and DECnet are both available to compile into the linux kernel, I'm not certain how one would set things up to switch all your apps to using tcp/ipx and so on.
the security goal on your lan is to frustrate someone who gains access to your wire. By using a non-standard transport layer (is that the right layer?it's been years) one could raise the bar just another little bit.
I agree that it would be more trouble than worth.