Dairy vs Almond: The Ongoing Battle over the 'Milk' Label (arstechnica.com)

According to the Oxford English Dictionary, the expression "mylke of almaundes", i.e. almond milk, was used in 1425. So I'd say the use of "milk" to refer to non-dairy products is fairly well established.