[Home]

Summary:ASTERISK-03595: [patch] rfc2833 DTMFs sent with bad timestamps
Reporter:m_frankied (m_frankied)Labels:
Date Opened:2005-02-26 14:45:42.000-0600Date Closed:2008-01-15 15:34:34.000-0600
Priority:MinorRegression?No
Status:Closed/CompleteComponents:Core/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) rtp_digitstamp_rev1.diff.txt
( 1) rtp.c.diff
( 2) rtpdump.gz
( 3) rtp-timestamp.diff.txt
( 4) senddigit.diff
( 5) rtp.c.spa3k.diff.txt
Description:I ran into an issue with the way asterisk sends rfc2833 DTMF events. I ran into the issue using 1.0.3 on BSD, but looking at the latest rtp.c the issue is still there. I have an asterisk forwarding RTP streams between a cisco ISDN/VOIP gateway and an intel HMP. When a caller (in the PSTN) enters some DTMFs in quick succession, asterisk sends the digits without voice packets in between. That is ok, except that it re-uses the timestamp of the first digit for the following digits.

So, if user presses '1' and then '2' on his phone, asterisk should send eg. 12 RTP telephone-event packets, 6 of them having one timestamp, 6 of them having another. The issue is that asterisk sends all 12 with the same timestamp.

The HMP box receives digits that claim to have occurred simultaneously and it ignores all but the first one.

the asterisk is configured to use dtmfmode=rfc2833.

The cause of the issue is the fact that ast_rtp_senddigit() in rtp.c uses rtp->lastts as the timestamp. lastts is the timestamp of the latest voice packet. If no voice packets are sent, lastts does not change. The patch below uses (lastts + number of ms elapsed since then) as timestamp for sending DTMF events. The diff is against rtp.c from cvs-head.

Note: discussion on asterisk-dev revealed that the patch below is nearly a rollback of the patch attached to issue 2928, so it should be tested whether fixing this does not break what 2928 fixed.
Comments:By: Brian West (bkw918) 2005-02-26 15:48:00.000-0600

This is really minor since it could be a protocol violation.
By: Brian West (bkw918) 2005-02-26 15:54:04.000-0600

get an RTP debug from both of your devices...  I wanna try to see if we can get away with using the same timestamp per digit where all three packets get the same timestamp... I wonder if that would flow correctly.  This is the part the RFC is not very clear on.

/b
By: Mark Spencer (markster) 2005-02-26 23:57:48.000-0600

Why is it not sufficient for the seqno to be increased?
By: Mark Spencer (markster) 2005-02-27 00:01:37.000-0600

It seems that having an increasing RTP seqno should be sufficient, so I feel this is a bug at the other end.  However, I've modified rtp.c in CVS head to increase by 20ms the timestamp if no audio has been sent in the mean time.  Please confirm this fixes the problem for you.  Thanks!
By: m_frankied (m_frankied) 2005-02-27 07:33:06.000-0600

This fix changes the issue from 'never works' to 'sometimes works'. Perhaps HMP finds the faked timestamps jump too much compared to each other, the surrounding G.711 and the real packet arrival times (the digits are sent ~200ms apart).
By: nick (nick) 2005-02-27 09:11:33.000-0600

Perhaps you might consider contacting Intel to determine what the HMP is expecting? Us playing "guess the brokenness" is probably not going to work out as well....

Nick
By: Brian West (bkw918) 2005-02-27 09:57:09.000-0600

When I initially fixed this everything I tested against only incremented the seqno not the timestamp.

Go figure.

/b
By: Mark Spencer (markster) 2005-02-27 10:46:12.000-0600

They should be arriving 20ms apart.  Anyway, I'm reverting the change I made and will contact Intel regarding this HMP issue.  How about using in-band or INFO based DTMF in the mean time?
By: Mark Spencer (markster) 2005-02-27 11:00:02.000-0600

Just for fun, by the way, go ahead and attach a trace of the (completely) failing RFC2833 transmission.  I've e-mailed my contact at Intel as well as the two authors listed in RFC2833, so we'll see if we get any magic back.
By: m_frankied (m_frankied) 2005-02-27 17:24:13.000-0600

Okay, pcap dump attached, from the POV of asterisk (ip x.156). first packet of first arriving digit from cisco (ip x.37) is frame 1083. first packet of first digit sent to HMP (ip x.154) is frame 1099.
By: m_frankied (m_frankied) 2005-03-14 15:16:49.000-0600

Intel says they expect the DTMF timings to conform to ITU standards for DTMF wrt. the duration of the tones and the duration of the silence between them (senders should put at least 40ms of silence between digits, receivers should expect at least 24ms of silence IIRC). That explains why constant timestamps don't work.

'Real' timestamps don't work reliably either. Consider the following scenario: suppose a digit arrives from ISDN GW at asterisk at T=0, and and the digit ends at T=100. Asterisk will send the digit to HMP at T=100 claiming a duration of 100. Suppose the next digit arrives at T=150 and it ends at T=200. Asterisk will send it at T=200. HMP will see 2 digits with no silence between them, so it will interpret the two digits as one digit.

The quick and dirty solution might be to use a timestamp like max(now,previous_ts+digit_length+40ms), and perhaps shortening the fixed digit length (currently 100ms) a bit to minimize cumulative effects.
By: Brian West (bkw918) 2005-03-14 21:17:01.000-0600

I don't think you're understanding this... from my view point and in more testing I find that the timestamp for the current DTMF NEVER increses and has in some cases increased between digits.. ie. dialing 1 then 2 then 3 then ... then 0.  Check rtp debug inbound from something like a cisco 7960 or sipura.. you'll notice that the seq increases per packet but the ts stays the exact same.

Press 1:
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21090, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21090, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21090, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21091, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21092, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21093, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21094, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21094, ts 329730255, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21094, ts 329730255, len 4)

Press 2:
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21169, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21169, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21169, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21170, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21171, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21172, ts 329742895, len 4)
Got RTP packet from 65.38.28.147:16414 (type 101, seq 21173, ts 329742895, len 4)

In any case I think you can't increase the timestamps from packet to packet on a single digit but its safe to do it interdigit wise.  Don't ask me why this sipura is acting like this but in either case it works with my TNT and everything else the way asterisk does it now.  

/b
By: m_frankied (m_frankied) 2005-03-15 14:51:00.000-0600

Bkw918, It seems my writing is ambiguous, so please accept the humble apologies of a sloppy non native speaker. We actually agree for at least 50% :-)

Your statement "you can't increase the timestamps from packet to packet on a single digit" is absolutely *true*.

You think so. I think so. We agree on this.

So given the fact that we agree on the above statement, when I say something like "the timestamp should vary in such and such a way", I *only* mean varying timestamps (in your words) "interdigit wise".

Now about this varying "interdigit wise": you say you have seen devices other than asterisk that keep the "interdigit" timestamp constant. I have never seen this myself and I have never seen a trace of it, so I cannot say whether it is true or not.

What I can say is that if such equipment exists, it will *definitely* not interoperate with intel. It will *probably* not interoperate with many other devices that translate rfc2833 to inband dtmf.

Therefore, an asterisk that honors the timing rules for DTMF would be more interoperable than an asterisk that doesn't, IMHO.
By: Brian West (bkw918) 2005-03-17 21:06:16.000-0600

Ya the problem is I have a TNT here that never increases the timestamps.. I have a sipura with the latest firmware that does.... so this is one of those WTF do we do now!

/b
By: Kevin P. Fleming (kpfleming) 2005-03-20 09:35:15.000-0600

Mark, it's been a few weeks, did you get any responses? This certainly seems to be a viable patch as it stands.
By: cherso (cherso) 2005-04-01 12:06:00.000-0600

I think the right example is something like this:
The RTP header timestamp is the same for all of the packets during
the event (0) and the duration increments by 400 timestamp units per
packet.

RTP seq#        RTP timestamp        duration
---------------        ---------------------        ------------
     0                        0                     400
     1                        0                     800
     2                        0                     1200
     3                        0                     1600

According to rfc2833 the marker bit is "true" just on the first packet
and the "end bit" must be true on the last 3 packets. The last 3 packets
should have the same timestamp and duration field (just a retrasmission)
My patch is doing that. I did not play attention to lastts, 0 is a good
value and last voice rtp timestamp is good too.
By: Kevin P. Fleming (kpfleming) 2005-04-03 19:46:10

OK, after _much_ discussion, we have concluded that Asterisk's behavior sending RTP for DTMF does appear to be conformant with the RFC. There is no need to be changing the duration for the packets being sent, since they are retransmissions, not extensions of the event.

However, in an attempt to work around the problem with the HMP, I have uploaded a patch (rtp_digitstamp_rev1.diff.txt) that ensures that all DTMF events (not the packets within an event) have unique, monotonically increasing timestamps.

Please test this patch and report back your findings.
By: Kevin P. Fleming (kpfleming) 2005-04-03 22:41:57

This fix has been committed to CVS, so please test CVS HEAD and report back if you continue to have to trouble with DTMF being sent by Asterisk.
By: m_frankied (m_frankied) 2005-04-04 03:33:13

rtp_digitstamp_rev1.diff.txt won't work because there is no 40ms gap between digits (see my bugnote of 03-14-05). n dtmfs in quick succession will look like one n*100ms tone after conversion from 2833 to inband and will be interpreted as such (at least on intel HMP).

Shortening the digit duration to below 60ms would probably make it work.
By: cherso (cherso) 2005-04-04 05:37:17

please try the senddigit.diff patch
By: m_frankied (m_frankied) 2005-04-04 05:58:59

cherso: I fail to see how senddigit.diff would help with this issue. It doesn't address the timing issues we've identified and it stretches the digit duration to 240ms, if I understand it correctly.
By: cherso (cherso) 2005-04-04 07:17:30

ast_rtp_senddigit is just one digit event function
Voice RTP packets are not affecting this function as you can read in the rfc2833
The rfc2833 told us that the timestamp could be 0 or fixed (when it has to be sent in the middle of voice rtp packets) and we have to increment the duration using 160 timestamp units every 20ms.
I can modify the senddigit.diff to get a variable duration of the event, but I think the patch is rfc2833 compliant.
I have not verified the asterisk dtmf receive and resent code, but I think asterisk is proxying the dtmf as received, so the sender should send a different timestamp for the 2nd dtmf.
I'm reading your rtp dump now... I'll post something later

edited on: 04-04-05 07:18
By: Kevin P. Fleming (kpfleming) 2005-04-04 11:05:43

M_frankied: If the HMP requires a gap between digits, and does not respect the 'end' flag in the RTP, then it is requiring something that the RFC does not specify. There is no language in the RFC mandating any particular timing of digits, or between digits. Generally, RFC-compliant systems are liberal in what they accept, especially for behavior that is left open to interpretation by the RFC. The current code sends distinct start/end events for each digit, which should be adequate for determining where each digit begins and ends.

cherso: Can you point to some language in RFC2833 that mandates that the duration field _must_ (not _should_) be increased for the second (and following) event packets?
By: cherso (cherso) 2005-04-04 16:22:39

2.2.1  Timestamp

  The RTP timestamp reflects the measurement point for the current
  packet.  The event duration described in Section 2.5 extends forwards
  from that time.  For events that span multiple RTP packets, the RTP
  timestamp identifies the beginning of the event, i.e., several RTP
  packets may carry the same timestamp.

2.2.2  Marker Bit

  The RTP marker bit indicates the beginning of a new event.

2.3.5  Duration Field

  The duration field indicates the duration of the event or subevent
  being reported, in timestamp units, expressed as an unsigned integer.
  For a non-zero value, the event or subevent began at the instant
  identified by the RTP timestamp and has so far lasted as long as
  indicated by this parameter.  The event may or may not have ended.
  If the event duration exceeds the maximum representable by the
  duration field, the event is split into several contiguous subevents
  as described below.

examples here
http://www1.ietf.org/mail-archive/working-groups/avt/current/msg01518.html

edited on: 04-04-05 16:24
By: Kevin P. Fleming (kpfleming) 2005-04-04 18:45:07

Those references are specifically for when there is an interpacket gap between the first four event packets; Asterisk does not have any gap. The six packets are all sent one right after the other, so there is no need to update the duration field between packets.
By: m_frankied (m_frankied) 2005-04-06 03:30:46

kpfleming: I don't represent intel so it is not really 'my fight', but don't you agree that rfc2833 wants to support bridging from analog to voip and vice versa and that sending rfc2833 digits that can't be represented in analog makes that difficult ?
By: Kevin P. Fleming (kpfleming) 2005-04-06 11:36:48

What does "can't be represented" mean here? The RTP that Asterisk sends for DTMF with the recent changes can very well be represented in analog mode perfectly... none of the digits overlap, and they are all marked as 100ms in duration. Requiring a _gap_ between digits is a completely different issue; the PSTN itself does not require a gap between digits, only that they do not overlap.
By: m_frankied (m_frankied) 2005-04-07 02:36:32

it's not hard for a signal detector to detect the sequence '12' without a gap, but detecting '11' without a gap would be physically impossible (it would look exactly like one long '1').
By: Kevin P. Fleming (kpfleming) 2005-04-13 01:22:12

That's true, I suppose. However, what circumstances have to occur to get Asterisk to send multiple DTMF sequentially with no gaps between them? Certainly they won't come in that way from any existing RTP device... Are we only talking about senddigit() calls that send more than one digit?
By: m_frankied (m_frankied) 2005-04-13 02:44:08

See bugnote 0023696 of mar-14 for a scenario where 'good' digits come into asterisk from another RTP device, but 'bad' digits are sent from asterisk as a result.
By: Kevin P. Fleming (kpfleming) 2005-04-20 11:57:23

OK, I agree with your analysis, and I've committed a fix to CVS HEAD to ensure that sequential DTMF digits are always spaced apart by a minimum of 20ms (according to the timestamps). Let's see if that solves the problem before extending it out farther... I'm not keen on stretching the incoming DTMF out any more than we have to.
By: Bayan Towfiq (implicit) 2005-05-12 02:55:41

reopened so that another patch can be posted by reporter
By: Enzo Michelangeli (enzo michelangeli) 2005-05-12 03:13:53

I asked for this bug to be reopened because the current code doesn't interoperate with the current version 2.0.13g of the Sipura SPA-3000 (and perhaps with other models as well). I also posted a message to http://voxilla.com/index.php?name=PNphpBB2&file=viewtopic&p=17641 .

The basic problem is that the SPA-3000 doesn't like to receive the three "Event end" packets with exactly the same timestamp as the three previous "Event start" packets: when that happens, instead of a tone it produces a sort of click. Incrementing the timestamp by one unit is sufficient to make the Sipura happy. RFC2833 seems to imply that the two timestamps should be different:

   [...]
   Alternatively, the receiver can
   start a tone and play it until it receives a packet with the
   "E" bit set, the next tone, distinguished by a different
   timestamp value or a given time period elapses.
   [...]

Anyway, the diff file rtp.c.spa3k.diff.txt I uploaded fixes the incompatibility.

Enzo


By: Kevin P. Fleming (kpfleming) 2005-05-14 23:56:18

That chunk of RFC you quoted specifically allows for Asterisk's current behavior. The tone should be played until _any_ of those conditions occur, so sending a packet with the 'E' bit set (regardless of the timestamp) is supposed to be adequate.

However, I don't see that this patch should cause any harm.
By: Kevin P. Fleming (kpfleming) 2005-05-15 00:00:59

Committed to CVS HEAD with minor mod, thanks!
By: Digium Subversion (svnbot) 2008-01-15 15:26:21.000-0600

Repository: asterisk
Revision: 5097

U   trunk/rtp.c

------------------------------------------------------------------------
r5097 | markster | 2008-01-15 15:26:21 -0600 (Tue, 15 Jan 2008) | 2 lines

Make sure we increase the timestamp between digits (bug ASTERISK-3595)

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=5097
By: Digium Subversion (svnbot) 2008-01-15 15:31:57.000-0600

Repository: asterisk
Revision: 5489

U   trunk/rtp.c

------------------------------------------------------------------------
r5489 | kpfleming | 2008-01-15 15:31:57 -0600 (Tue, 15 Jan 2008) | 2 lines

ensure that sequential DTMF sent via RTP has some 'dead air' between digits (bug ASTERISK-3595)

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=5489
By: Digium Subversion (svnbot) 2008-01-15 15:34:34.000-0600

Repository: asterisk
Revision: 5666

U   trunk/rtp.c

------------------------------------------------------------------------
r5666 | kpfleming | 2008-01-15 15:34:34 -0600 (Tue, 15 Jan 2008) | 2 lines

fix RTP DTMF for Sipura SPA-3000 (bug ASTERISK-3595, yet again)

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=5666