"any application that defines an FEC
that does not take measures to prevent the values 0x4 and 0x6 from
occurring in the first nibble of the payload may be subject to IP
ECMP and thus having their flows take multiple paths and arriving
with considerable jitter and possibly out of order."
That's not enough context I feel for most readers. I'm quoting a bit more here which I think will better explain the issue:
"In the early days of MPLS, the payload was almost exclusively IP. Even today the overwhelming majority of carried traffic remains IP. Providers of MPLS equipment sought to continue this IP ECMP behavior. As shown above, it is not possible to know whether the payload of an MPLS packet is IP at every place where IP ECMP needs to be performed. Thus vendors have taken the liberty of guessing the payload. By inspecting the first nibble beyond the label stack, existing equipment infers that a packet is not IPv4 or IPv6 if the value of the nibble (where the IP version number would be found) is not 0x4 or 0x6 respectively. Most deployed LSRs will treat a packet whose first nibble is equal to 0x4 as if the payload were IPv4 for purposes of IP ECMP.
A consequence of this is that any application that defines an FEC that does not take measures to prevent the values 0x4 and 0x6 from occurring in the first nibble of the payload may be subject to IP ECMP and thus having their flows take multiple paths and arriving with considerable jitter and possibly out of order."
> "In the early days of MPLS, the payload was almost exclusively IP. Even today the overwhelming majority of carried traffic remains IP."
You can pretty much carry "Any Transport over MPLS" (AToM). As mentioned, IP is the most common protocol, of course, but you can "tunnel" almost anything you can think of over an MPLS path: Ethernet, ATM, PPP, HDLC, SONET, etc. Basically, you dump a frame into one side and it comes out the other unmodified -- regardless of the network in between.
As noted in the linked e-mail, an ISP providing (layer 2) transport has (little to) no control over the payload so there's nothing they can do to prevent running into this issue.
https://tools.ietf.org/html/bcp128
"any application that defines an FEC that does not take measures to prevent the values 0x4 and 0x6 from occurring in the first nibble of the payload may be subject to IP ECMP and thus having their flows take multiple paths and arriving with considerable jitter and possibly out of order."