Some journals require upright Greek letters for constants as opposed to
variables, for example $\upi$ when used to denote 3.14159... as opposed
to a variable ($\pi$ is sometimes used to denote parallax in astronomy,
for instance). (Some journals define \upi as "upright pi", \upi as
"upright i", and so on.)

I certainly agree that LABELS should be upright (though they are usually
Latin not Greek) and not italic to distinguish them from variables, e.g.
$T_{mathrm{eff}}$ for effective temperature or $\rho_{textrm{g}}$ for
gas density, say, as opposed to $G_{\mu\nu}$ where $\mu$ and $\nu$ are
not constants but variables.

And it is not just Greek letters. For example, e for the Euler number
or i for the square root of -1 should also not be in math italic, to
distinguish them from variables. I tend to agree with that as well.
Also, units should be upright, e.g. 5 m and not $5m$ for 5 metres.

On the other hand, I have never seen the gravitational constant $G$,
which is even by definition a constant and not a variable, written
upright. Ditto for the Hubble constant $H$ and so on.

Or is there a difference between mathematical constants and physical
constants?

Perhaps because standard (La)TeX provides Greek letters only in math
italic, upright Greek letters are less common than upright Latin
letters, even when used in the same way (labels, units, symbols which
are not variables).

When writing for a specific journal, one usually has to follow the house
style. However, if there is no rule, I prefer to do what is generally
deemed to be correct. What is generally deemed to be correct here?