Minutes of the 18th NMRG meeting INRIA/LORIA, Nancy, France 30-31 July 2005 Minutes: Aiko Pras Participants: - Michael Alexander (WU Vienna, Austria) - Remi Badonnel (LORIA-INRIA, France) - Vincent Cridlig (LORIA-INRIA, France) - Olivier Festor (LORIA-INRIA, France) - James Hong (Postech, Korea) - Christian Hoene (TU Berlin, Germany) - Azita Kia (Cisco Systems, USA) - Abdelkader Lahmadi (LORIA-INRIA, France) - Saverio Niccolini (NEC Europe, Germany) - Amy Pendleton (Nortel Networks, USA) - David Perkins (SNMPInfo, USA) - Aiko Pras (University of Twente, the Netherlands) - Juergen Quittek (NEC Europe, Germany) - Dan Romascanu (Avaya, Israel) - Juergen Schoenwaelder (International University Bremen, Germany) - Henning Schulzrinne (Columbia University, USA) - Radu State (LORIA-INRIA, France) Agenda: Saturday (2005-07-30) - Welcome (Juergen Schoenwaelder, Olivier Festor) - Real-time Application Quality of Service Monitoring (Dan Romascanu) - RTP Control Protocol Extended Reports (Amy Pendleton) - RTP, RTCP XR and SIP MIB Modules (Dan Romascanu) - SIP Service Quality Reporting (Amy Pendleton) - Pre-provisioning, Template, Individual and Per-Subscriber Provisioning for VoIP Services (Michael Alexander) - Management and QoS for VoIP (Henry Sinnreich / Juergen Schoenwaelder) - User-oriented Management of VoIP Applications (Henning Schulzrinne) - Service Provider VoIP OSS - Lessons Learned (Azita Kia) - Calculation of Speech Quality by Aggregating the Impacts of Individual Frame Losses (Christian Hoene) - VoIP Security Threat Analysis (Saverio Niccolini) - VoIP Fuzzer, Cracker and Spammer - incl. demo (Olivier Festor, Radu State) Sunday (2005-07-31) - Discussion: Identify main research/engineering questions to be addressed - Discussion: Develop plans how the research/engineering questions could be addressed - Wrap Up (Juergen Schoenwaelder, Olivier Festor) ======================================================================= 10:15: Welcome by Juergen Schoenwaelder. Roll call. The structure of this meeting differs from traditional NMRG meetings, in the sense that the first day is entirely reserved for presentations. Many discussions will therefore be postponed until the second day. Saturday morning started with four presentations that covered more or less identical topics: monitoring VoIP quality at end devices. 10:25 Real-time Application Quality of Service Monitoring (RAQMON) Dan Romascanu (Avaya, Israel) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/raqmon.pdf RAQMON is being developed by the IETF RMON WG. There are three Internet Drafts, covering the following topics: Framework, PDUs and MIB. The idea is that every end device includes a RAQMON data source, which sends RAQMON PDUs to a collector. These PDUs are used by the collector to maintain "a kind of RMON MIB", which can be queried by managers via SNMP. Information between data source and collector is exchanged using TCP or SNMP notifications. Many parameters have been defined, like addresses of the communicating parties, number of packets, delay, jitter, loss and CPU utilization; for a complete overview see the slides. There are currently two different code bases, and probably 10 applications that use these bases. The question was raised for whom RAQMON was developed. Dan answered that RAQMON may be interesting for providers running VoIP or video distribution networks. A discussion followed on the fact that RAQMON should be implemented in all end systems. Since there may be millions of them, scalability might become problematic. 11:00 RTP Control Protocol Extended Reports (RTCP XR) Amy Pendleton (Nortel, USA) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/rtcp-xr.pdf Control Protocol Extended Reports (RTCP XR) is being developed by the IETF Audio/Video Transport (AVT) WG. Its definition can be found in RFC 3611, which is at Proposed Standard level. The principle goal is facilitate monitoring / measurements on a per call basis. The provided metrics include: loss, jitter, high/low signal level, echo, noise and delay; for a complete overview see the slides. Possible application scenarios include fault management, provisioning and accounting. The presentation included a number of measurement results. One of the interesting conclusions was that it is not very useful to know only average loss / discard figures; it is much more important to know the distribution. To determine such distribution, a four-state Markov Packet Loss model is proposed. There are two states for Gaps, which indicate minor problems, and two for Bursts, which indicate serious problem. A window of 16 packets is needed to maintain these states. Henning questioned the use of four states, and assumed that two would be sufficient. Amy continued her presentation with explaining how signal level problems, like clipping, can be detected, as well as Echo. The last part of her presentation concentrated on the design philosophy behind RTCP XR, as well as a framework for VoIP performance management (see slides). The framework allowed the use of probes to capture VoIP related data. In the subsequent discussion the problem was raised of how to analyze such data in case of encryption, as well as the level at which VoIP quality measurements should take place. Christian believed that such measurements should be performed at a much higher level, yielding easy to use metrics. He said that there are many research papers that demonstrate that VoIP quality can be measured with little knowledge of network parameters. Henning did not agree on this, and also Azita said that we still have a long way to go before we understand VoIP quality measurement like we understand traditional PSTN telephone quality measurements. 11:45 RTP, RTCP XR and SIP MIB Modules Dan Romascanu (Avaya, Israel) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/voip-mibs.pdf Dan gave an overview of VoIP related MIB activities within the IETF. Three MIB modules are under development; these modules are not overlapping, but complementary. The first module is the RTP MIB, which is being revised and currently and Internet Draft. This module is intended for end systems, and keeps track of call history. Aiko asked if this module could also be used for "data retention" purposes. Dan wasn't sure, because the MIBs structure may be too complex for this purpose. The second module is the RTCP XR MIB, and is based on the extended reports technology as defined in RFC 3611 Proposed Standard). The MIB is structured around the following RTP concepts: Session, Sender and Receiver. It can be implemented in RTP Host Systems, as well as Intermediate Systems. The data model includes History as well as Call Quality parameters. The third module is the SIP MIB, which is still an Internet Draft and not on standards track yet. The module can be implemented in all kinds of SIP entities (see RFC 3261), thus within User Agents, as well as Servers (Proxy, Redirect and Registrar). The supported operations include status monitoring, protocol statistics and configuration of notifications. The structure and indexing of the SIP MIB is based on the Network-Services MIB (RFC 2788). It should be noted that all three modules can be used for monitoring purposes only; they can not be used for VoIP configuration purposes. For that purpose there are other tools and protocols, however (see URL in Dan's slides). The level of deployment of the three MIB modules is unclear; in fact the meeting participants questioned that SNMP will become the technology of choice for VoIP management. 12:10 SIP Service Quality Reporting Amy Pendleton (Nortel, USA) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/sip-qr.pdf The last presentation of this morning session focused on event notification capabilities in SIP. These notifications are not related to the SIP MIB, but are separate messages with a textual syntax following SIP conventions (which is not XML). The work is performed within the SIPPING WG and still at Internet Draft level; in February 2005 it went for last call, but there were many comments that a new draft needs to be created. 12:30 Lunch Break 15:00 Management and QoS for VoIP Henry Sinnreich (pulver.com) and Juergen Schoenwaelder (IUB, Germany) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/qos.pdf Juergen made two provocative statements to trigger discussion. First, QoS is not an issue to worry about for VoIP in general, and for VoIP management in particular. Second, P2P self organizing networks will provide the highest possible availability for VoIP services. Juergen argued that specific management mechanisms for VoIP may not be necessary, but that we need more generic management mechanisms, which are also useful for other applications. In addition, we do not need a centralized "operator", but can have a P2P network to coordinate and exchange management data (for his precise reasoning: see the slides). Although some agreed with his statements, others strongly disagreed. A discussion than started on where you need to measure: in end systems, or elsewhere. Another discussion popped up whether perceived quality does not largely depend on the codecs being used; Skype uses high quality codecs and many therefore perceive Skype as a high quality VoIP system. Some participants had doubts how easy it is to replace the common G.7** codecs by higher quality codecs, since most of the technology is patented and may not become available as open source codecs. 15:40 Preprovisioning, Template, Individual and Per-Subscriber Provisioning for VoIP Services Michael Alexander (WU Vienna, Austria) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/voip-prov.pdf Provisioning a VoIP subscriber entails setting a variety of per-line and per-customer group parameters in several devices and Operation Support Systems (OSS). In mixed PSTN/VoIP Systems with local PSTN gateways, at the minimum 3 devices need to be provisioned; for in-network VoIP the count is two plus OSS - with at the minimum inventory management being affected. The resulting element and OS management overhead is considerable especially when considering that CPE provisioning is expected to be performed primarily by operators going forward. Michael presented various approaches to lower the management burden of provisioning VoIP circuits. When utilizing pre-provisioning and template-based provisioning in addition to per-subscriber provisioning, he claimed that the management burden could potentially be reduced. The advantages of the former two approaches in conjunction with per-circuit provisioning were discussed and some notions on needs requirements for VoIP management protocols were derived. 16:40 User-oriented Management of VoIP Applications Henning Schulzrinne (Columbia University, USA) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/dyswis.pdf Henning explained that VoIP can fail for any number of reasons, from low-level connectivity problems to signaling failure, NAT issues, packet loss and jitter as well as subtle end-system-related problems that have the same effect as network problems. Often, these problems are transient and cannot be reproduced. While the user sees one application, the end user application, user OS, home network, access network, voice service provider (proxy operator) and the remote party all need to cooperate to complete a call. Thus, there are probably half a dozen parties that can be blamed, in mutual finger pointing, if something goes wrong. Although Henning argued that you need a relatively small number of "health measurement" tests to detect such problems, traditional network management tools are of only limited help in this environment. Henning's proposal was therefore to introduce a "Do you see what I see" mechanism, which allows the various parties to query neighboring systems if they experience similar problems. For example, a system could ask a neighboring system if it has connectivity, can pass the NAT, and if it is able to get a certain DNS address. In the discussion the participants agreed that such "Do you see what I see" mechanism will have certain implications for security, which require further research. 17:05 Service Provider VoIP OSS - Lessons Learned Azita Kia (Cisco Systems, USA) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/voip-oss.pdf Azita presented a short (3 slides) but nice historical overview of VoIP management. She summarized some lessons learned in 10 years of VoIP deployments, and identified some areas of standardization that might help to improve these deployments. One of the recommendations is to work further on accounting; this recommendation was followed by a discussion what items we expect operators will charge for. Some argued that all kind of supplementary services, like Calling Line Identification Presentation, will be separately charged for. Others argued that the investment for operators could be too high to charge for this. In addition, customers may not be willing to pay for this and move to alternative providers. 17:20 Calculation of Speech Quality by Aggregating the Impacts of Individual Frame Losses Christian Hoene (TU Berlin, Germany) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/speech.pdf Christian gave a similar presentation as its previous IWQoS presentation in Passau; his findings were part of his PhD work. His message is that packet loss doesn't say much; it depends on which packets get lost. There are less important packets, and more important packets. Christian gave a demonstration, in which he simulated 35% packet loss, and made clear that if the most important packets get lost, you can not understand anything. If only the less important packets get lost, voice is still understandable. Christian's results may be useful for WIFI environments at home, or if you want to save on energy of handhelds. Christian expected that the backbone network will be good enough and does not need to do differentiate between important and less important packets. During the discussion, it became clear that there is currently no good online mechanism to classify VoIP packets into important and unimportant ones and that this is a subject for further research. 17:45 VoIP Security Threat Analysis Saverio Niccolini (NEC Europe, Germany) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/voip-sec.pdf Saverio gave an overview of all kind of security threats for VoIP (see slides for details). These threats come from the fact that the signaling is sent using the same network as the multimedia data and that the traffic is not encrypted. In his talk Saverio also presented possible countermeasures and the pros and cons of some currently proposed solutions. Saverio's work is quite similar to the work of VoIP security Alliance, although his work is less advanced. Saverio referenced the SIPPING WG's Internet Draft on SIP SPAM (http://ietf.org/internet-drafts/draft-ietf-sipping-spam-01.txt), and told that NEC applied for a patent on this topic. Saverio's intention is to submit this work as paper to IEEE Network Magazine Special Issue on Securing Voice over IP. 18:00 VoIP Fuzzer, Cracker and Spammer (incl. demo) Olivier Festor, Radu State (LORIA-INRIA, France) http://www.ibr.cs.tu-bs.de/projects/nmrg/meetings/2005/nancy/voip-fuzzer.pdf The first day of the meeting ended with a demo on management of an Asterix VoIP system, using INRIA/LORIA's netconf implementation. This demo was followed by a demo of a security assessment tool, called "fuzzy packet". This tool manipulates and generates data messages by injecting and capturing packets into the network. It used "random" users and passwords (generated by script), and is able to inject SPAM into an ongoing call. Quite interesting! 19:00 Leaving for Dinner ======================================================================= July 31 - start at 9:30 Dan: There is a difference in understanding what VoIP management really means. Two extremes: 1) alternative / replacement of existing POTS -> VoIP should comply to same regulatory requirements as VoIP (legal interception / CDRs) -> has many technical / management implications. Same reliability & QoS requirements as POTS. 2) Skype. Dan explained the need in some environments to have CDRs. Aiko said that you do not need SNMP like agent capabilities in end systems, but you can use Netflow to create CDRs. Henning and Amy said that Netflow captures data at a too low level, you have IP addresses and not end-user (SIP) information. Dan went on discussing about performance management, like we have for the POTS (route optimization, trunk provisioning etc). He said he is not aware of much work in this area (within standardization). Aiko asked what the management requirements would be if the the second extreme (=Skype) would be successful. Dan did not know if / what they did to manage a network like things. Henning said they are primarily monitoring / gathering statistics, but not active interfering. Henning said that we're not clearly defining what we mean with management. Amy said Skype is doing a lot of management within the application, like buffer management / auto gain control. Question by Juergen: What are the challenging problems we have to work on? Henning made an inventory of possible management activities: provisioning, SLA management, CDRs (AAA, billing, fraud management, dimensioning), QoS Statistics (Real time, utilization), security (SBC, firewalls, IDS, patch management), real-time alarms. Juergen said we should not list all possible management functions. Challenges for research include: security impact in management (like intercept of RTCP-XR), automation versus human planning, scalability (data reduction, filtering), what are the SLA metrics (problems in case of multiple providers), modeling. Henning concluded that these are not very specific for VoIP. Is there something that is specific for VoIP? Discussion started on what level you should define measures. On a per call basis (fine granularities), or on a monthly (for example) basis? Aiko believed that metrics are not much different between VoIP and other application (like video). Michael argued that you need more modeling. Currently the device has a big MIB, and you don't know which objects you need to modify if you want to achieve a specific effect. For TDM networks, to name an example, we have such models, but not for IP. In addition, metrics equivalent to errored seconds in TDM systems would be useful to have for VoIP networks. Dave said that this is a well-known problem, but that within the IETF we have not been able to do so, since you need for this the operators, and they do not know. Meeting did not agree on the feasibility / possible success of such modeling work. We agreed that we should have a small number of "objects". Henning put on the white board possible protocol / technology needs. For discovery you have several approaches, like LLDP, Bonjour and SLP. It was noted that LLDP is based on the ENTITY-MIB (RFC 2737). Juergen said that we are making the same mistake as in the past. We think in terms of the manager-agent model, and believe that we're done if the agents are able to provide all kind of information. The actual problem, however, is how the "magic" / intelligence performed within the manager looks like. Aiko said that he wants to get rid of such centralized components as much as possible, let's put more intelligence within the devices, like the "do you see what I see" approach, outlined by Henning the day before. Michael would like to see work on SIP specific FCAPS. Dan proposed, as example, to discuss a bit more performance. Latency was mentioned as one of the interesting metrics. The question was if you have to measure this specific for SIP, but for all kind of applications. Juergen collected proposals for further work: 1) SIP-enabled VoIP FCAPS Data or Information Model (could lead to an informational RFC) 2) VoIP metric framework (at a higher level than MIBs) which explains which metrics and which access/distribution mechanisms are being defined. 3) Fault incident reports (readable by machines) which provide a structured high-level explanation of a fault experienced by VoIP entity. 4) Fault isolation / diagnostic procedures that can be used locally or called remotely to produce fault incident reports. 5) Management implications of VoIP business models (enterprises, operators who own the network, pure VoIP operators which do not have a network and non-standard compliant operators, like Skype) How to proceed with respect to the various proposals: 1) Juergen does not yet sufficiently understand this idea, and asked Michael to create a document that explains this idea a bit better and provides examples. Initiative taken by Michael, Dan is also interested. 2) VoIP metric framework could be an article for IEEE Communications Magazine or IEEE Surveys and Tutorials). Initiative taken by Amy, Dan and others. 3) Fault incident report. Initiative taken by Henning. 4) Fault isolation / Diagnostic procedures. Initiative taken by Henning. 5) Management implications of VoIP business models. Initiative taken by Aiko. Meeting closed Sunday at 12:05