Bug 406

Created attachment 298 [details]
sample file for testing the bug

Simulation example illustrating the GlobalRouteManager problem.

Yes, global routing only works in the pre-simulation stage at present.  I will try to provide some kind of workaround for you later tonight.  

For the longer-term cleaner fix, in general, if we want to support calling PopulateRoutingTables() at any time in the simulation, we need to be able to keep track of the routes that were previously injected, so they could be removed.  One way would be to tag each route somehow so the routes iterated could be examined for the tag.  Another way would be to store these routes in a dedicated global routing object that is distinct from the static routing ("manual" routing) object that users may want to hand-enter default routes, and to call Flush() on this routing object.  Right now, those routes are mixed together in Ipv4StaticRouting so we do not want to just delete every route we find in that object.

(In reply to comment #2)
> Yes, global routing only works in the pre-simulation stage at present.  I will
> try to provide some kind of workaround for you later tonight.  
> 
> For the longer-term cleaner fix, in general, if we want to support calling
> PopulateRoutingTables() at any time in the simulation, we need to be able to
> keep track of the routes that were previously injected, so they could be
> removed.  One way would be to tag each route somehow so the routes iterated
> could be examined for the tag.  Another way would be to store these routes in a
> dedicated global routing object that is distinct from the static routing
> ("manual" routing) object that users may want to hand-enter default routes, and
> to call Flush() on this routing object.  Right now, those routes are mixed
> together in Ipv4StaticRouting so we do not want to just delete every route we
> find in that object.

Erm, the behavior I was expecting was that PopulateRoutingTables would reset the routing tables of all nodes it is invoked on. i.e., it would empty the forwarding tables before filling them.

Created attachment 299 [details]
patch to initialize routes

After posting the bug, we tried a couple of different things. It appears that calling the InitializeRoutes() method during the simulation does solve the no route to the host problem. Since InitializeRoutes is a private method, we added a new public method (PromptInitializeRoutes) that basically call the InitializeRoutes and then we schedule this new function as follows:

Simulator::Schedule (Seconds (1.75001),GlobalRouteManager::PromptInitializeRoutes

Attached is a patch that we used. I am not sure how sound this strategy is?

However, this does not help with the case where there are multiple paths between a source destination pair. When one of the paths becomes unavailable, we are not able to make the simulator use the alternative path with the global routing manager.

Created attachment 300 [details]
Updated patch to initialize routes

Updated the patch with corrections... There was a typo error...

> 
> Attached is a patch that we used. I am not sure how sound this strategy is?
> 
> However, this does not help with the case where there are multiple paths
> between a source destination pair. When one of the paths becomes unavailable,
> we are not able to make the simulator use the alternative path with the global
> routing manager.
> 

The reason the patch will not completely work is that it doesn't delete the existing routes.  Try adding a loop that deletes everything first, before reinitializing things.

for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
  {
    ipv4->RemoveRoute (i)
  }

(In reply to comment #3)
> (In reply to comment #2)
> > Yes, global routing only works in the pre-simulation stage at present.  I will
> > try to provide some kind of workaround for you later tonight.  
> > 
> > For the longer-term cleaner fix, in general, if we want to support calling
> > PopulateRoutingTables() at any time in the simulation, we need to be able to
> > keep track of the routes that were previously injected, so they could be
> > removed.  One way would be to tag each route somehow so the routes iterated
> > could be examined for the tag.  Another way would be to store these routes in a
> > dedicated global routing object that is distinct from the static routing
> > ("manual" routing) object that users may want to hand-enter default routes, and
> > to call Flush() on this routing object.  Right now, those routes are mixed
> > together in Ipv4StaticRouting so we do not want to just delete every route we
> > find in that object.
> 
> Erm, the behavior I was expecting was that PopulateRoutingTables would reset
> the routing tables of all nodes it is invoked on. i.e., it would empty the
> forwarding tables before filling them.
> 

That would be the easiest behavior to code, but would tend to make it harder for users to mix hand-crafted route installation (such as adding defaults to portions of the topology, such as an ad hoc routing subnet) with global route installation.  Such users would have to keep re-entering their static routing entries each time the global routing changed.

(In reply to comment #7)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Yes, global routing only works in the pre-simulation stage at present.  I will
> > > try to provide some kind of workaround for you later tonight.  
> > > 
> > > For the longer-term cleaner fix, in general, if we want to support calling
> > > PopulateRoutingTables() at any time in the simulation, we need to be able to
> > > keep track of the routes that were previously injected, so they could be
> > > removed.  One way would be to tag each route somehow so the routes iterated
> > > could be examined for the tag.  Another way would be to store these routes in a
> > > dedicated global routing object that is distinct from the static routing
> > > ("manual" routing) object that users may want to hand-enter default routes, and
> > > to call Flush() on this routing object.  Right now, those routes are mixed
> > > together in Ipv4StaticRouting so we do not want to just delete every route we
> > > find in that object.
> > 
> > Erm, the behavior I was expecting was that PopulateRoutingTables would reset
> > the routing tables of all nodes it is invoked on. i.e., it would empty the
> > forwarding tables before filling them.
> > 
> 
> That would be the easiest behavior to code, but would tend to make it harder
> for users to mix hand-crafted route installation (such as adding defaults to
> portions of the topology, such as an ad hoc routing subnet) with global route
> installation.  Such users would have to keep re-entering their static routing
> entries each time the global routing changed.

how about ClearRoutingTables as an extra function ?

(In reply to comment #6)
> > 
> > Attached is a patch that we used. I am not sure how sound this strategy is?
> > 
> > However, this does not help with the case where there are multiple paths
> > between a source destination pair. When one of the paths becomes unavailable,
> > we are not able to make the simulator use the alternative path with the global
> > routing manager.
> > 
> 
> The reason the patch will not completely work is that it doesn't delete the
> existing routes.  Try adding a loop that deletes everything first, before
> reinitializing things.
> 
> for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
>   {
>     ipv4->RemoveRoute (i)
>   }
> 

We have couple of questions:

1-I am not sure where to implement above code partly due to lack of complete picture of GlobalRouting implementation. 

2-Once the routes are removed, I guess we can rebuild the routes on all nodes with BuildGlobalRoutingDatabase () and InitializeRoutes () methods. Is this accurate?

(In reply to comment #6)
> > 
> > Attached is a patch that we used. I am not sure how sound this strategy is?
> > 
> > However, this does not help with the case where there are multiple paths
> > between a source destination pair. When one of the paths becomes unavailable,
> > we are not able to make the simulator use the alternative path with the global
> > routing manager.
> > 
> 
> The reason the patch will not completely work is that it doesn't delete the
> existing routes.  Try adding a loop that deletes everything first, before
> reinitializing things.
> 
> for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
>   {
>     ipv4->RemoveRoute (i)
>   }
> 

We were able to delete the existing routes as mentioned above. However, it appears that recalculating the routes using BuildGlobalRoutingDatabase and InitializeRoutes results in the exact same routes being calculated. In other words, the system does not recognize the fact that a certain link (ipv4 interface) is down in its routing calculation.

I double checked this bug by doing a simple experiment. I SetDown an ipv4 interface immediately after creating the link. I observed that the global routing manager ignores this fact and goes on to choose the disabled link in its routing calculations. 

Is there a way to include a check in the routing calculation to see if the interface of the link is up or down? Perhaps a call to the IsUp method of ipv4 class before proceeding with routing calculation?

(In reply to comment #10)
> (In reply to comment #6)
> > > 
> > > Attached is a patch that we used. I am not sure how sound this strategy is?
> > > 
> > > However, this does not help with the case where there are multiple paths
> > > between a source destination pair. When one of the paths becomes unavailable,
> > > we are not able to make the simulator use the alternative path with the global
> > > routing manager.
> > > 
> > 
> > The reason the patch will not completely work is that it doesn't delete the
> > existing routes.  Try adding a loop that deletes everything first, before
> > reinitializing things.
> > 
> > for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
> >   {
> >     ipv4->RemoveRoute (i)
> >   }
> > 
> 
> We were able to delete the existing routes as mentioned above. However, it
> appears that recalculating the routes using BuildGlobalRoutingDatabase and
> InitializeRoutes results in the exact same routes being calculated. In other
> words, the system does not recognize the fact that a certain link (ipv4
> interface) is down in its routing calculation.
> 
> I double checked this bug by doing a simple experiment. I SetDown an ipv4
> interface immediately after creating the link. I observed that the global
> routing manager ignores this fact and goes on to choose the disabled link in
> its routing calculations. 
> 
> Is there a way to include a check in the routing calculation to see if the
> interface of the link is up or down? Perhaps a call to the IsUp method of ipv4
> class before proceeding with routing calculation?

I believe that the current behavior of the code is correct, that is, the global routing code should setup all routes, regardless of the ipv4 up or down state. It seems to me that what should happen is one or a mix of:
1) make the per-node route lookup be aware of the ipv4 link state. that is, route lookup should not return a route for an ipv4 interface which is down. Instead, it should a route with potentially a lower metric but which at least is up
2) make sure that the global route setup configures multiple routes with different metrics using multiple interfaces if there are multiple interfaces on a node such that the route lookup in (1) can fallback to different interfaces if some of them are down.




However, it should probably also setup routes such that the ip route lookup in each node's forwarding table returns a working route when various ipv4 interfaces are down. 

(In reply to comment #9)
> (In reply to comment #6)
> > > 
> > > Attached is a patch that we used. I am not sure how sound this strategy is?
> > > 
> > > However, this does not help with the case where there are multiple paths
> > > between a source destination pair. When one of the paths becomes unavailable,
> > > we are not able to make the simulator use the alternative path with the global
> > > routing manager.
> > > 
> > 
> > The reason the patch will not completely work is that it doesn't delete the
> > existing routes.  Try adding a loop that deletes everything first, before
> > reinitializing things.
> > 
> > for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
> >   {
> >     ipv4->RemoveRoute (i)
> >   }
> > 
> 
> We have couple of questions:
> 
> 1-I am not sure where to implement above code partly due to lack of complete
> picture of GlobalRouting implementation. 

For some basic overview material, see Section 9.9 of the manual:
http://www.nsnam.org/docs/release/manual.html#SEC100

>>> When one of the paths becomes unavailable,
> > > we are not able to make the simulator use the alternative path with the global
> > > routing manager.

The algorithm in the global routing manager is Dijkstra's SPF algorithm, without equal-cost multipath.  So, there are no alternate paths that are precomputed.  With the existing code (which was ported from quagga's OSPFv2 implementation) one would have to recompute routes whenever there is a topology change or link change, to find an alternate route.

> 
> 2-Once the routes are removed, I guess we can rebuild the routes on all nodes
> with BuildGlobalRoutingDatabase () and InitializeRoutes () methods. Is this
> accurate?
> 

I believe so, but I haven't tested that in particular.

(In reply to comment #10)
> (In reply to comment #6)
> > > 
> > > Attached is a patch that we used. I am not sure how sound this strategy is?
> > > 
> > > However, this does not help with the case where there are multiple paths
> > > between a source destination pair. When one of the paths becomes unavailable,
> > > we are not able to make the simulator use the alternative path with the global
> > > routing manager.
> > > 
> > 
> > The reason the patch will not completely work is that it doesn't delete the
> > existing routes.  Try adding a loop that deletes everything first, before
> > reinitializing things.
> > 
> > for (uint32_t i = 0; i < ipv4->GetNRoutes(); i++)
> >   {
> >     ipv4->RemoveRoute (i)
> >   }
> > 
> 
> We were able to delete the existing routes as mentioned above. However, it
> appears that recalculating the routes using BuildGlobalRoutingDatabase and
> InitializeRoutes results in the exact same routes being calculated. In other
> words, the system does not recognize the fact that a certain link (ipv4
> interface) is down in its routing calculation.
> 
> I double checked this bug by doing a simple experiment. I SetDown an ipv4
> interface immediately after creating the link. I observed that the global
> routing manager ignores this fact and goes on to choose the disabled link in
> its routing calculations. 
> 
> Is there a way to include a check in the routing calculation to see if the
> interface of the link is up or down? Perhaps a call to the IsUp method of ipv4
> class before proceeding with routing calculation?
> 
Yes, we could add a check to look at the Ipv4Interface state of the interface.  I think that we need to do this check to support the dynamic behavior you are looking for.

> > 
> > I double checked this bug by doing a simple experiment. I SetDown an ipv4
> > interface immediately after creating the link. I observed that the global
> > routing manager ignores this fact and goes on to choose the disabled link in
> > its routing calculations. 
> > 
> > Is there a way to include a check in the routing calculation to see if the
> > interface of the link is up or down? Perhaps a call to the IsUp method of ipv4
> > class before proceeding with routing calculation?
> 
> I believe that the current behavior of the code is correct, that is, the global
> routing code should setup all routes, regardless of the ipv4 up or down state.
> It seems to me that what should happen is one or a mix of:
> 1) make the per-node route lookup be aware of the ipv4 link state. that is,
> route lookup should not return a route for an ipv4 interface which is down.
> Instead, it should a route with potentially a lower metric but which at least
> is up
> 2) make sure that the global route setup configures multiple routes with
> different metrics using multiple interfaces if there are multiple interfaces on
> a node such that the route lookup in (1) can fallback to different interfaces
> if some of them are down.
> 
There is only one route calculated to each destination.  This is not a multipath routing algorithm.
> 
> 
> 
> However, it should probably also setup routes such that the ip route lookup in
> each node's forwarding table returns a working route when various ipv4
> interfaces are down. 
> 

If the topology changes, routes at each node should be recomputed.  If routers autonomously make decisions to recalculate routes, or suppress telling their neighbors about link up/down events, there will likely be routing loops or black holes.  This particular algorithm avoids loops by enforcing a consistent view of the topology at each node.

Here is an API proposal to allow a user to delete and recompute routes in the middle of a simulation.  Note that this doesn't include any forthcoming API expansion that will allow users to selectively enable portions of the topology to be covered by the global routing (see bug 66).

Current public API:
------------------
GlobalRouteManager::PopulateRoutingTables()

Proposed new public API:
------------------------
GlobalRouteManager::PopulateRoutingTables()  /* same as before */

/**
 * @brief Remove all routes that were previously installed in a prior call
 * to either PopulateRoutingTables() or RecomputeRoutingTables(), and 
 * add a new set of routes.  This method does not change the set of nodes
 * over which GlobalRouting is being used, but it will dynamically update
 * its representation of the global topology before recomputing routes.
 * Users must first call PopulateRoutingTables() and then may subsequently
 * call RecomputeRoutingTables() at any later time in the simulation.
 */
GlobalRouteManager::RecomputeRoutingTables()

I will try to upload a new patch to fix this, with an example program, sometime tomorrow.  

Thanks for the comments and ideas.  We will be looking forward to the patch.

In regards to the GlobalRoutingManager, we understand that it is for the static routing. For the scenarios (P2P links) that we were testing we tried OLSR as well. It looks like once a link (i.e. interface) is down, the traffic is not being carried via the alternate path (we are giving enough time for the OLSR to recalculate the routes and etc.). We just wanted to bring up that subject, but for that issue we will open a new bug. 

We assumed that OLSR should work both in wired and wireless scenarios. Is this correct?

Thanks for the comments and ideas.  We will be looking forward to the patch.

In regards to the GlobalRoutingManager, we understand that it is for the static routing. For the scenarios (P2P links) that we were testing we tried OLSR as well. It looks like once a link (i.e. interface) is down, the traffic is not being carried via the alternate path (we are giving enough time for the OLSR to recalculate the routes and etc.). We just wanted to bring up that subject, but for that issue we will open a new bug. 

We assumed that OLSR should work both in wired and wireless scenarios. Is this correct?

(In reply to comment #18)
> Thanks for the comments and ideas.  We will be looking forward to the patch.

The patch is getting close I think, but turned out to be more involved than I first thought.

> 
> In regards to the GlobalRoutingManager, we understand that it is for the static
> routing. For the scenarios (P2P links) that we were testing we tried OLSR as
> well. It looks like once a link (i.e. interface) is down, the traffic is not
> being carried via the alternate path (we are giving enough time for the OLSR to
> recalculate the routes and etc.). We just wanted to bring up that subject, but
> for that issue we will open a new bug. 
> 
> We assumed that OLSR should work both in wired and wireless scenarios. Is this
> correct?
> 

Yes, although another user found an issue with how MID messages are working (bug 407).  Please open another bug if you think that OLSR is not handling down interfaces properly.

Created attachment 309 [details]
patch to enable dynamic global routing

This patch can be applied against ns-3-dev:

cd ns-3-dev/
patch -p1 < bug406.patch

Please look over the (next attachment) test program, and then see whether it helps your use case also.

Created attachment 310 [details]
test program

This test program does some manipulation of the topology and uses the dynamic features of global routing to change the global routing tables.

//  Order of events
//  At pre-simulation time, configure global routes.  Shortest path from
//  n1 to n6 is via the direct point-to-point link
//  At time 1s, start CBR traffic flow from n1 to n6
//  At time 2s, set the n1 point-to-point interface to down
//  At time 3s, call RecomputeGlobalRoutes() and traffic will
//    start flowing again on the alternate path
//  At time 4s, re-enable the n1/n6 interface to up.  Will not change routing
//  At time 5s, call RecomputeGlobalRoutes() and traffic will start flowing 
//    again on the original path

(In reply to comment #21)
> Created an attachment (id=310) [details]
> test program
> 
> This test program does some manipulation of the topology and uses the dynamic
> features of global routing to change the global routing tables.
> 
> //  Order of events
> //  At pre-simulation time, configure global routes.  Shortest path from
> //  n1 to n6 is via the direct point-to-point link
> //  At time 1s, start CBR traffic flow from n1 to n6
> //  At time 2s, set the n1 point-to-point interface to down
> //  At time 3s, call RecomputeGlobalRoutes() and traffic will
> //    start flowing again on the alternate path
> //  At time 4s, re-enable the n1/n6 interface to up.  Will not change routing
> //  At time 5s, call RecomputeGlobalRoutes() and traffic will start flowing 
> //    again on the original path
> 

First of all, let me thank you for your prompt and helpful support. I installed the patch and the above test scenario works as you mention.

However, we found three potential problems as shown with the following scenarios:

Scenario 1:

Problem Summary:
An alternate path is used only if the sink is attached to the ipv4 interface on the alternate path. 

Testing:
In the above mentioned example, connect the sink application to i1i6 instead of attaching to i5i6. Replace i5i6.GetAddress (1) with i1i6.GetAddress (1) in line-133. No traffic flows between 3sec and 4 secs despite of an available alternate link.

Scenario 2:

Problem Summary:
If the interface to which the sink is attached goes down, all traffic stops even if there exists an alternate route to the destination node.

Order of events: (changes marked with '-->')
//  At pre-simulation time, configure global routes.  Shortest path from
//  n1 to n6 is via the direct point-to-point link
//  At time 1s, start CBR traffic flow from n1 to n6
//--->  At time 2s, set down the point-to-point interface *1* on node *n6*. T
//  The Traffic flow stops
//--->  At time 3s, call RecomputeGlobalRoutes() and traffic does *NOT*
//    flow again the alternate path
//  At time 4s, re-enable the n1/n6 interface to up.  Will not change routing
//---> The traffic will start flowing on the original path whether or not you Recomute GlobalRoutes
//  At time 5s, call RecomputeGlobalRoutes() and traffic will continue to flow 


Scenario 3:

Problem Summary:
A link does not go down unless both the interfaces on either side of the link are setdown, with the exception of source interface.
 
Testing:
In the example that you gave, setdown ipv4 interface on node 6 corresponding to link 1-6. If no further steps are taken, the traffic continues to flow on the this failed link (link 1-6). 

Please let me know if you want me to post the files for each of the above scenarios.

(In reply to comment #22)

> 
> However, we found three potential problems as shown with the following
> scenarios:
> 
> Scenario 1:
> 
> Problem Summary:
> An alternate path is used only if the sink is attached to the ipv4 interface on
> the alternate path. 
> 
> Testing:
> In the above mentioned example, connect the sink application to i1i6 instead of
> attaching to i5i6. Replace i5i6.GetAddress (1) with i1i6.GetAddress (1) in
> line-133. No traffic flows between 3sec and 4 secs despite of an available
> alternate link.

This is an outcome of the limitation that is documented in bug 66.  Specifically, when you put the sink there, the algorithm should process the interface as a Type 3 (stub) network link but we didn't finish off the support in the Dijkstra to do the stub networks yet.

I can raise the priority of fixing that.  I have some code to fix it that needs some cleanup and testing.

> 
> Scenario 2:
> 
> Problem Summary:
> If the interface to which the sink is attached goes down, all traffic stops
> even if there exists an alternate route to the destination node.
> 
> Order of events: (changes marked with '-->')
> //  At pre-simulation time, configure global routes.  Shortest path from
> //  n1 to n6 is via the direct point-to-point link
> //  At time 1s, start CBR traffic flow from n1 to n6
> //--->  At time 2s, set down the point-to-point interface *1* on node *n6*. T
> //  The Traffic flow stops
> //--->  At time 3s, call RecomputeGlobalRoutes() and traffic does *NOT*
> //    flow again the alternate path
> //  At time 4s, re-enable the n1/n6 interface to up.  Will not change routing
> //---> The traffic will start flowing on the original path whether or not you
> Recomute GlobalRoutes
> //  At time 5s, call RecomputeGlobalRoutes() and traffic will continue to flow 
> 

This one is debatable whether it is correct or incorrect behavior.  If the interface is down, is the IP address associated with that interface available still on the node, via other interfaces?  If this is a weak ES model (RFC 1122), perhaps, but it probably varies from system to system whether it works or not.

> 
> Scenario 3:
> 
> Problem Summary:
> A link does not go down unless both the interfaces on either side of the link
> are setdown, with the exception of source interface.
> 
> Testing:
> In the example that you gave, setdown ipv4 interface on node 6 corresponding to
> link 1-6. If no further steps are taken, the traffic continues to flow on the
> this failed link (link 1-6). 

Indeed, this is a bug in Ipv4L3Protocol::Receive().  IPv4L3Protocol::Receive() should check its Ipv4Interface state and drop the packet.  The reason that the packet is even sent at all on the interface is that, on the outbound interface, there is no globally computed route but the link is still "up" from the perspective of the sending node and the address matches the prefix on the p2p link.

> 
> Please let me know if you want me to post the files for each of the above
> scenarios.
> 

Not necessary, I was able to reproduce each one.  Thank you for the detailed report.  Sorry this is causing you problems; I'll expedite a new patch.

Created attachment 318 [details]
patch for dynamic global routing

Revised patch.

Created attachment 319 [details]
test program

revised test program

(In reply to comment #23)
> (In reply to comment #22)
> 
> > 
> > However, we found three potential problems as shown with the following
> > scenarios:
> > 
> > Scenario 1:
> > 
> > Problem Summary:
> > An alternate path is used only if the sink is attached to the ipv4 interface on
> > the alternate path. 
> > 
> > Testing:
> > In the above mentioned example, connect the sink application to i1i6 instead of
> > attaching to i5i6. Replace i5i6.GetAddress (1) with i1i6.GetAddress (1) in
> > line-133. No traffic flows between 3sec and 4 secs despite of an available
> > alternate link.
> 
> This is an outcome of the limitation that is documented in bug 66. 
> Specifically, when you put the sink there, the algorithm should process the
> interface as a Type 3 (stub) network link but we didn't finish off the support
> in the Dijkstra to do the stub networks yet.
> 
> I can raise the priority of fixing that.  I have some code to fix it that needs
> some cleanup and testing.

The latest patch fixes the above, and it is tested during time 11-15 of the attached test program.

> 
> > 
> > Scenario 2:
> > 
> > Problem Summary:
> > If the interface to which the sink is attached goes down, all traffic stops
> > even if there exists an alternate route to the destination node.
> > 
> > Order of events: (changes marked with '-->')
> > //  At pre-simulation time, configure global routes.  Shortest path from
> > //  n1 to n6 is via the direct point-to-point link
> > //  At time 1s, start CBR traffic flow from n1 to n6
> > //--->  At time 2s, set down the point-to-point interface *1* on node *n6*. T
> > //  The Traffic flow stops
> > //--->  At time 3s, call RecomputeGlobalRoutes() and traffic does *NOT*
> > //    flow again the alternate path
> > //  At time 4s, re-enable the n1/n6 interface to up.  Will not change routing
> > //---> The traffic will start flowing on the original path whether or not you
> > Recomute GlobalRoutes
> > //  At time 5s, call RecomputeGlobalRoutes() and traffic will continue to flow 
> > 
> 
> This one is debatable whether it is correct or incorrect behavior.  If the
> interface is down, is the IP address associated with that interface available
> still on the node, via other interfaces?  If this is a weak ES model (RFC
> 1122), perhaps, but it probably varies from system to system whether it works
> or not.

I did not add support for this, since the associated IPv4 interface is in a down state (and the address could be considered to be inactive and unavailable via a different interface).

> 
> > 
> > Scenario 3:
> > 
> > Problem Summary:
> > A link does not go down unless both the interfaces on either side of the link
> > are setdown, with the exception of source interface.
> > 
> > Testing:
> > In the example that you gave, setdown ipv4 interface on node 6 corresponding to
> > link 1-6. If no further steps are taken, the traffic continues to flow on the
> > this failed link (link 1-6). 
> 
> Indeed, this is a bug in Ipv4L3Protocol::Receive().  IPv4L3Protocol::Receive()
> should check its Ipv4Interface state and drop the packet.  The reason that the
> packet is even sent at all on the interface is that, on the outbound interface,
> there is no globally computed route but the link is still "up" from the
> perspective of the sending node and the address matches the prefix on the p2p
> link.

This is fixed in the current patch, and tested during time 6-9 seconds in the test program.

fixed in changesets ec65107df095, 34908804c029, 7658bcc28d8d

I tested the patch using the ns3-dev. It looks like you already patched/sourced the development release with the additional scenarios we notified, so I didn't have to patch the previous ns-dev release (after the first global routing patch you delivered).

After we mentioned the scenarios, we realized that the second scenario is a border line and ns-3 operates like real life network. We are in agreement on that too.

I tested the patch with some scenarios we had as well, and it looks like it is working.

Is it a good assumption that these fixes will be implemented in the ns-3.3 release targeted for Dec. 17?

Again, thank you very much for your effort and support trying to get this fixed. 

Best regards,

(In reply to comment #28)
> I tested the patch using the ns3-dev. It looks like you already patched/sourced
> the development release with the additional scenarios we notified, so I didn't
> have to patch the previous ns-dev release (after the first global routing patch
> you delivered).

Yes, the ns-3.3 release candidate and ns-3-dev both have this patch.  
> 
> After we mentioned the scenarios, we realized that the second scenario is a
> border line and ns-3 operates like real life network. We are in agreement on
> that too.
> 
> I tested the patch with some scenarios we had as well, and it looks like it is
> working.
> 
> Is it a good assumption that these fixes will be implemented in the ns-3.3
> release targeted for Dec. 17?

Yes.  Plus, I will put the test program in the attachments into the examples directory, but I didn't want to disrupt the release process with that yet.

> 
> Again, thank you very much for your effort and support trying to get this
> fixed. 
> 

Thanks for your comments; please file reports for other things that you find in the future.