Bug 1929

Hi,

you are partially right. The first call to RouteOutput is to find the route *and* the source address.

When a node has more than one IP address, then it must choose the "best" one, and the routing protocol is the only thing able to decide.

From there onward, all the RouteOutput calls will contain also the source address.

To solve the problem one could:
1) have a separate function (not RouteOutput) to probe for the source IP address (nevertheless, it must be performed by routing) or
2) live with it.

(In reply to Morteza Kheirkhah from comment #0)
> When ECMP per flow is used with TCP the result is not quite right.
> The reason is that TcpL4Protocol::SendPacket() need to find a route to send
> its packets, so it query the routing to find one. To do so it create a dummy
> packet header and pass it to Ipv4GlobalRouting::RouteOutput(). The problem
> here is that source IP address that is essential for ECMP per flow to work
> is not added to the dummy header passed on RouteOutput(). Current ECMP per
> flow is based on sum of five tuple (srcIp, dstIp, protocol, sPort, dPort). 
> 
> Please let me know if I made any wrong assumption. Otherwise I would be
> happy to write a patch to fix this.
> 

I agree with Tommaso, I don't see a problem with the RouteOutput API for what you want to do.

Morteza, I believe that your main issue in getting this to work is that Ipv4GlobalRouting does not have support for host-based ECMP, correct?

Hi Tommaso,

(In reply to Tommaso Pecorella from comment #1)
> Hi,
> 
> you are partially right. The first call to RouteOutput is to find the route
> *and* the source address.
> 
> When a node has more than one IP address, then it must choose the "best"
> one, and the routing protocol is the only thing able to decide.
> 
> From there onward, all the RouteOutput calls will contain also the source
> address.
> 
> To solve the problem one could:
> 1) have a separate function (not RouteOutput) to probe for the source IP
> address (nevertheless, it must be performed by routing) or
> 2) live with it.

I agree!

Hi Tom,

(In reply to Tom Henderson from comment #2)
> (In reply to Morteza Kheirkhah from comment #0)
> > When ECMP per flow is used with TCP the result is not quite right.
> > The reason is that TcpL4Protocol::SendPacket() need to find a route to send
> > its packets, so it query the routing to find one. To do so it create a dummy
> > packet header and pass it to Ipv4GlobalRouting::RouteOutput(). The problem
> > here is that source IP address that is essential for ECMP per flow to work
> > is not added to the dummy header passed on RouteOutput(). Current ECMP per
> > flow is based on sum of five tuple (srcIp, dstIp, protocol, sPort, dPort). 
> > 
> > Please let me know if I made any wrong assumption. Otherwise I would be
> > happy to write a patch to fix this.
> > 
> 
> I agree with Tommaso, I don't see a problem with the RouteOutput API for
> what you want to do.
> 
> Morteza, I believe that your main issue in getting this to work is that
> Ipv4GlobalRouting does not have support for host-based ECMP, correct?

Correct!!

A scenario that I'm trying to explore here is that when an end-host is multi-homed, FlowEcmpRouting is enabled, there are equal cost paths to destination, and also RouteOutput() is used to retrieve the best sourceIp address (output NetDevice) for each flow at connection setup. 

In this scenario FlowECMPRouting would choose a random route for connection initiator (there is no best route here, all routes are equal cost). For finding the best route, ECMP needs to sum the five tuple of packet header. But because of the lack of sourceIp address the default value of "102.102.102.102" is used, which has been initialised at object creation of each Ipv4Address. This value would be used in every connection setup! Clearly this simply hurt to the avalanche effect of ECMP algorithm especially when the only difference between different flows is their sourceIp address (common scenario in MultiPath-TCP). I guess we can mitigate this issue to some extend by setting sourceIp address of the found route to all future call to the RouteOutput() from TcpL4Protocol::SendPacket() by passing dummy Ipv4Header that has the sourceIp address as well as destination address. So in this case ECMP can achieve better path splitting among the flows. Furthermore, this would not has any negative effect to any existing functionality of Ipv4GlobalRouting at all, it only be used when ECMP is used.

The issue here is that the actual route where the packets are sent out might be different from the first route provided by ECMP. This might not be serious issue since Ipv4GlobalRouting is implemented based on weak end-host model!

Another very important scenario to consider here is that when a transport protocol has explicit knowledge of its available source ip addresses and want to split its flows throughout them (common scenario in MultiPath-TCP in data centres). So if the TcpL4Protocol::SendPacket() does not supply the sourceIp address of its flow, when it query the Ipv4GlobalRouting for finding a route, then all flows would send their packets via the *same* route since ECMP would failed to split them at all.

Those are my thoughts, Please correct me if I am wrong.

(In reply to Morteza Kheirkhah from comment #4)
> Hi Tom,
> 
> (In reply to Tom Henderson from comment #2)
> > (In reply to Morteza Kheirkhah from comment #0)
> > > When ECMP per flow is used with TCP the result is not quite right.
> > > The reason is that TcpL4Protocol::SendPacket() need to find a route to send
> > > its packets, so it query the routing to find one. To do so it create a dummy
> > > packet header and pass it to Ipv4GlobalRouting::RouteOutput(). The problem
> > > here is that source IP address that is essential for ECMP per flow to work
> > > is not added to the dummy header passed on RouteOutput(). Current ECMP per
> > > flow is based on sum of five tuple (srcIp, dstIp, protocol, sPort, dPort). 
> > > 
> > > Please let me know if I made any wrong assumption. Otherwise I would be
> > > happy to write a patch to fix this.
> > > 
> > 
> > I agree with Tommaso, I don't see a problem with the RouteOutput API for
> > what you want to do.
> > 
> > Morteza, I believe that your main issue in getting this to work is that
> > Ipv4GlobalRouting does not have support for host-based ECMP, correct?
> 
> Correct!!
> 
> A scenario that I'm trying to explore here is that when an end-host is
> multi-homed, FlowEcmpRouting is enabled, there are equal cost paths to
> destination, and also RouteOutput() is used to retrieve the best sourceIp
> address (output NetDevice) for each flow at connection setup. 
> 
> In this scenario FlowECMPRouting would choose a random route for connection
> initiator (there is no best route here, all routes are equal cost). For
> finding the best route, ECMP needs to sum the five tuple of packet header.
> But because of the lack of sourceIp address the default value of
> "102.102.102.102" is used, which has been initialised at object creation of
> each Ipv4Address. This value would be used in every connection setup!
> Clearly this simply hurt to the avalanche effect of ECMP algorithm
> especially when the only difference between different flows is their
> sourceIp address (common scenario in MultiPath-TCP). I guess we can mitigate
> this issue to some extend by setting sourceIp address of the found route to
> all future call to the RouteOutput() from TcpL4Protocol::SendPacket() by
> passing dummy Ipv4Header that has the sourceIp address as well as
> destination address. So in this case ECMP can achieve better path splitting
> among the flows. Furthermore, this would not has any negative effect to any
> existing functionality of Ipv4GlobalRouting at all, it only be used when
> ECMP is used.
> 
> The issue here is that the actual route where the packets are sent out might
> be different from the first route provided by ECMP. This might not be
> serious issue since Ipv4GlobalRouting is implemented based on weak end-host
> model!
> 
> Another very important scenario to consider here is that when a transport
> protocol has explicit knowledge of its available source ip addresses and
> want to split its flows throughout them (common scenario in MultiPath-TCP in
> data centres). So if the TcpL4Protocol::SendPacket() does not supply the
> sourceIp address of its flow, when it query the Ipv4GlobalRouting for
> finding a route, then all flows would send their packets via the *same*
> route since ECMP would failed to split them at all.
> 
> Those are my thoughts, Please correct me if I am wrong.

Closing the bug, as it's not a bug.

About how to solve your issue, you have to detect the non-initialized source header and manage it.
Remember that, once the route has been chosen, the source address is in the routing table entry that is being returned by RouteInput.

T.

*** Bug 2137 has been marked as a duplicate of this bug. ***

(In reply to Tommaso Pecorella from comment #6)
> *** Bug 2137 has been marked as a duplicate of this bug. ***

Hi Tommaso,

Many thanks for reopening this bug, I really appreciate it! I hope we will have interesting discussions about MPTCP soon.

Regards,
Morteza

Back then I completely misunderstood the issue.
TCP knows its source address, but it doesn't pass it to the routing protocol (as it should).
It is up to the routing protocol to consider the source address and "honor" it by using an output interface that is compatible with the said source address.

changeset 11434	317ae8a858d8