|
Bugzilla – Full Text Bug Listing |
| Summary: | FlowMon needs to be documented (Manual) | ||
|---|---|---|---|
| Product: | ns-3 | Reporter: | Tom Henderson <tomh> |
| Component: | flow-monitor | Assignee: | Tommaso Pecorella <tommaso.pecorella> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | ns-bugs, ville.saarinen |
| Priority: | P5 | ||
| Version: | ns-3-dev | ||
| Hardware: | All | ||
| OS: | All | ||
| Attachments: |
Patch
documentation |
||
I'll take a look. However, if this is the issue, then it's a bug in FlowMon. From a quick look at it, the issue is: 1) the packet is being dropped due to DROP_FRAGMENT_TIMEOUT. 2) the packet is not complete (of course) 3) there's not enough data in the fragment blob, and the UDP header is truncated 4) boom Note that the issue might arise with TCP as well. This case can be safely handled by checking that the "packet" it's long enough to contain the header, i.e., UDP or TCP. However, and this is the most important thing... what shall we do in this case ? It's impossible to read the right source and destination ports, so to make things short, FlowMon will *not* report the packet in m_flowMonitor->ReportDrop(). Or, we can assign 2 "fake" source and address ports (maybe 0) and record the drop. What's the best solution ? Created attachment 1723 [details]
Patch
The patch simply skips the packet classification if the packet doesn't carry a valid L4 protocol header (i.e., too short).
It might happens with fragments and/or ICMP errors.
Since I do not have a (non) working example of the bug, I can't reproduce it or confirm that the patch is correct. On the other hand, it does make sense according to the bug trace reported.
Leaving the bug as patch pending since I can't confirm 100% that the bug is gone. Waiting a second opinion to apply it. (In reply to Tommaso Pecorella from comment #3) > Leaving the bug as patch pending since I can't confirm 100% that the bug is > gone. Waiting a second opinion to apply it. I don't think the patch is robust to all fragmentation cases. A fragment may exist that is larger than the UDP header, yet the deserialized header will yield invalid values. I would rather see that the fragmentation offset is checked to be 0 before trying to deserialize the header. That will probably cause segfaults to stop occurring, but may still result in accounting problems in the flow monitor. My read of the code in ipv4-flow-probe.cc is that if classify is false, then the segment will not be counted. If the first segment is lost, the remaining segments will be unaccounted for. Is this the intended behavior? If so, should be clarified. A unit test around this is needed. Suggested unit test is to configure a 68-byte MTU on a point-to-point link and send a large UDP datagram (e.g. 200 bytes) through it, and write test cases to force a drop of the first, middle, or last segment using an error model. e.g. sender (has 68-byte MTU) -> intermediate hop -> final hop deploy a list error model that can cause first, second, or third segment to be dropped, and configure flow monitor on the intermediate hop outbound interface. Check that flow monitor accounting is correct. You're right about triple checking the fragmentation (and the docs). Unfortunately the guy that reported this is unresponsive, so I don't have the original bugged program. I'll try to build some tests to check what's happening when fragmentation occurs. About what to do and the accounting, I don't think it's a dramatic issue. FlowMon already take into account "lost" packets and purges the stats for vanished stuff. I'm not sure if the Tag added to the packets could help on this, I'll try with a double-MTU network. T. Finally I was able to reproduce the bug. And I fixed it. changeset 10467:c5bf751f8d4e (reopening for clarifications)
Thanks for the patch, but I had a few questions:
2.51 + // This allows to read the ports even on fragmented packets
2.52 + // not carrying a full TCP or UDP header.
what fragmentation case exists for which a full TCP or UDP header is not present? IPv4 minimum MTU is 68 bytes, and we are not sending IPv4 options.
Can it be documented in both the flow monitor sphinx documentation and the flow monitor doxygen that IPv4 Flow Monitor is incompatible with IPv4 fragmentation, and will ignore fragments leading to incorrect counts? Perhaps this is most clearly stated that it supports only non-fragmented TCP and UDP datagrams for IPv4 and ignores everything else (fragments, other transport protocols, tunnels, etc.).
I feel that long-term this fragmentation limitation should be fixed (probably with tags); I'd like to suggest it as a project idea (feature request) rather than dropping the subject. Agree? If so, are any test cases around (or even example program that can be easily modified for small MTU) that can exhibit the problem?
For record-keeping, the thread on the list is archived here:
http://mailman.isi.edu/pipermail/ns-developers/2013-November/011531.html
(In reply to Tom Henderson from comment #7) > (reopening for clarifications) > > Thanks for the patch, but I had a few questions: > > 2.51 + // This allows to read the ports even on fragmented packets > 2.52 + // not carrying a full TCP or UDP header. > > what fragmentation case exists for which a full TCP or UDP header is not > present? IPv4 minimum MTU is 68 bytes, and we are not sending IPv4 options. You're right. It's a bug. #1810 :P On the other hand, an ICMP only carry 8 bytes of the original payload. Hence... > Can it be documented in both the flow monitor sphinx documentation and the > flow monitor doxygen that IPv4 Flow Monitor is incompatible with IPv4 > fragmentation, and will ignore fragments leading to incorrect counts? > Perhaps this is most clearly stated that it supports only non-fragmented TCP > and UDP datagrams for IPv4 and ignores everything else (fragments, other > transport protocols, tunnels, etc.). Might I rephrase in "Can be FlowMon be documented"? Yes, it might and it should. As soon as I'll have some time, I'll write a complete documentation for it. Right now FlowMon have the Doxygen "clean", but little or no sphinx. > I feel that long-term this fragmentation limitation should be fixed > (probably with tags); I'd like to suggest it as a project idea (feature > request) rather than dropping the subject. Agree? If so, are any test > cases around (or even example program that can be easily modified for small > MTU) that can exhibit the problem? Agree, see bug# 1808. Example... tcp-large-transfer.cc. Simply modify one of the p2p links MTU. > For record-keeping, the thread on the list is archived here: > http://mailman.isi.edu/pipermail/ns-developers/2013-November/011531.html I'm gonna leave this bug open for the documentation part, but changing its attributes as they should be. *** Bug 1748 has been marked as a duplicate of this bug. *** Created attachment 1804 [details]
documentation
FlowMonitor manual (was attached to the wrong bug#)
Documentation added in changeset 10693:f9eddc26f100 Closing the bug. |
Reported on ns-3-users; from the backtrace, it may be that a fragment is passed to the flow classifier that assumes a UDP header can be popped. ou can find the backtrace below: #0 0xb7fdd424 in __kernel_vsyscall () #1 0xb51231df in raise () from /lib/i386-linux-gnu/libc.so.6 #2 0xb5126825 in abort () from /lib/i386-linux-gnu/libc.so.6 #3 0xb53b113d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/i386-linux-gnu/libstdc++.so.6 #4 0xb53aeed3 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6 #5 0xb53aef0f in std::terminate() () from /usr/lib/i386-linux-gnu/libstdc++.so.6 #6 0xb785b6a5 in ns3::Buffer::Iterator::Next (this=0xbfffd744, delta=2) at ./ns3/buffer.h:680 #7 0xb6c06525 in ns3::UdpHeader::Deserialize (this=0xbfffd930, start=...) at ../src/internet/model/udp-header.cc:198 #8 0xb6330864 in ns3::Packet::PeekHeader (this=0x875c6a8, header=...) at ../src/network/model/packet.cc:296 #9 0xb6f787c2 in ns3::Ipv4FlowClassifier::Classify (this=0x82f6750, ipHeader=..., ipPayload=..., out_flowId=0xbfffd9cc, out_packetId=0xbfffd9d0) at ../src/flow-monitor/model/ipv4-flow-classifier.cc:122 #10 0xb6f7b60f in ns3::Ipv4FlowProbe::DropLogger (this=0x82fe2c8, ipHeader=..., ipPayload=..., reason=ns3::Ipv4L3Protocol::DROP_FRAGMENT_TIMEOUT, ipv4=..., ifIndex=1) at ../src/flow-monitor/model/ipv4-flow-probe.cc:270 #11 0xb6f7eb3f in ns3::MemPtrCallbackImpl<ns3::Ptr<ns3::Ipv4FlowProbe>, void (ns3::Ipv4FlowProbe::*)(ns3::Ipv4Header const&, ns3::Ptr<ns3::Packet const>, ns3::Ipv4L3Protocol::DropReason, ns3::Ptr<ns3::Ipv4>, unsigned int), void, ns3::Ipv4Header const&, ns3::Ptr<ns3::Packet const>, ns3::Ipv4L3Protocol::DropReason, ns3::Ptr<ns3::Ipv4>, unsigned int, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (this=0x82feaf8, a1=..., a2=..., a3=ns3::Ipv4L3Protocol::DROP_FRAGMENT_TIMEOUT, a4=..., a5=1) at ./ns3/callback.h:235 #12 0xb6c28e6a in ns3::Callback<void, ns3::Ipv4Header const&, ns3::Ptr<ns3::Packet const>, ns3::Ipv4L3Protocol::DropReason, ns3::Ptr<ns3::Ipv4>, unsigned int, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (this=0x82fe338, a1=..., a2=..., a3=ns3::Ipv4L3Protocol::DROP_FRAGMENT_TIMEOUT, a4=..., a5=1) at ./ns3/callback.h:422 #13 0xb6c25859 in ns3::TracedCallback<ns3::Ipv4Header const&, ns3::Ptr<ns3::Packet const>, ns3::Ipv4L3Protocol::DropReason, ns3::Ptr<ns3::Ipv4>, unsigned int, ns3::empty, ns3::empty, ns3::empty>::operator() (this=0x82cca8c, a1=..., a2=..., a3=ns3::Ipv4L3Protocol::DROP_FRAGMENT_TIMEOUT, a4=..., a5=1) at ./ns3/traced-callback.h:240 #14 0xb6c216a4 in ns3::Ipv4L3Protocol::HandleFragmentsTimeout (this=0x82cca30, key=..., ipHeader=..., iif=1) at ../src/internet/model/ipv4-l3-protocol.cc:1428 #15 0xb6c21c22 in ns3::EventMemberImpl3::Notify (this=0x937d360) at ./ns3/make-event.h:188 #16 0xb6122b6d in ns3::EventImpl::Invoke (this=0x937d360) at ../src/core/model/event-impl.cc:45 #17 0xb61277b4 in ns3::DefaultSimulatorImpl::ProcessOneEvent (this=0x81041c8) at ../src/core/model/default-simulator-impl.cc:140 #18 0xb6127b4d in ns3::DefaultSimulatorImpl::Run (this=0x81041c8) at ../src/core/model/default-simulator-impl.cc:193 #19 0xb61238eb in ns3::Simulator::Run () at ../src/core/model/simulator.cc:160 #20 0x0806eef4 in MyExperiment::Run (this=0xbfffee54) at ../scratch/my-experiment.cc:706 #21 0x0806a484 in main (argc=11, argv=0xbffff034) at ../scratch/my-experiment.cc:236