Here's one for Http payload Container or TCP experts!

Working with a modem. It issues status reports in the form of about 270 bytes of text (text/html) via HTTP.

Although the modem responds perfectly to Firefox, the mbed HTTPClient connects, issues the GET, receives a 200 (OK) from the modem - but it captures nothing of the payload in the HTTPText container. The container appears to be empty.

HTTPResponse issues no error (0), and http responds 200 OK.

The code is simply the HTTPClientExample from the cookbook, altered for fixed IP:

http://mbed.org/users/RodColeman/programs/ModemClientReader/lx39kt

.

The funny thing is that, if I collect the payload with Firefox, then paste it into a text file, and place it on a website.... mbed reads it perfectly, and returns the payload. There's nothing funny about the payload, just text, see below.

Is there something I am missing about payload collection?

Other things I have tried:

- 3 different modems; - ordinary txt files on the web - changing the timeout - Segundos Netservices library

Since it works perfectly if you read it with Firefox. I used wireshark to see the traffic, when my Linux laptop reads the modem.

Here's the Wireshark report of the TCP packets when the modem is queried:

TCP data arrives in two frames. First, at t=16.6056 one containing the HTTP/1.0 200 (OK response):

0000 60 eb 69 0a 61 57 00 a0 bc 2f 20 d2 08 00 45 00 `.i.aW.. ./ ...E. 0010 00 45 82 ff 40 00 40 06 6e 5e c0 a8 64 01 c0 a8 .E..@.@. n^..d... 0020 64 03 00 50 87 44 2e e2 44 83 bb e0 67 3e 80 18 d..P.D.. D...g>.. 0030 01 ad 96 d2 00 00 01 01 08 0a 00 03 c6 ab 00 24 ........ .......$ 0040 f0 1e 48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f ..HTTP/1 .0 200 O 0050 4b 0d 0a K..

next, the actual payload is delivered, in a frame of 265 bytes, at time t = 16.6064:

0000 60 eb 69 0a 61 57 00 a0 bc 2f 20 d2 08 00 45 00 `.i.aW.. ./ ...E. 0010 00 fb 83 00 40 00 40 06 6d a7 c0 a8 64 01 c0 a8 ....@.@. m...d... 0020 64 03 00 50 87 44 2e e2 44 94 bb e0 67 3e 80 18 d..P.D.. D...g>.. 0030 01 ad 01 93 00 00 01 01 08 0a 00 03 c6 ac 00 24 ........ .......$ 0040 f0 2a 43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 .*Conten t-type: 0050 74 65 78 74 2f 68 74 6d 6c 0a 0a 2f 69 6d 61 67 text/htm l../imag 0060 65 73 2f 50 6f 69 6e 74 69 6e 67 5f 49 63 6f 6e es/Point ing_Icon 0070 5f 30 30 31 5f 43 6f 75 72 73 65 5f 46 61 64 65 _001_Cou rse_Fade 0080 2e 70 6e 67 23 23 2f 69 6d 61 67 65 73 2f 50 6f .png/i mages/Po 0090 69 6e 74 69 6e 67 5f 49 63 6f 6e 5f 30 30 31 5f inting_I con_001_ 00a0 46 69 6e 65 5f 46 61 64 65 2e 70 6e 67 23 23 2f Fine_Fad e.png/ 00b0 69 6d 61 67 65 73 2f 50 6f 69 6e 74 69 6e 67 5f images/P ointing_ 00c0 49 63 6f 6e 5f 30 30 31 5f 43 6f 6d 70 6c 65 74 Icon_001 _Complet 00d0 65 5f 46 61 64 65 2e 70 6e 67 23 23 2d 31 32 23 e_Fade.p ng-12# 00e0 23 31 30 25 23 23 30 2e 30 30 23 23 30 2e 30 30 #10%0. 000.00 00f0 23 23 30 23 23 30 2e 30 30 30 30 30 30 23 23 30 00.0 000000 0100 2e 30 30 30 30 30 30 23 23 .000000# #

Wireshark shows the reassembled TCP to look as might be expected:

0000 48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f 4b 0d HTTP/1.0 200 OK. 0010 0a 43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 74 .Content -type: t 0020 65 78 74 2f 68 74 6d 6c 0a 0a 2f 69 6d 61 67 65 ext/html ../image 0030 73 2f 50 6f 69 6e 74 69 6e 67 5f 49 63 6f 6e 5f s/Pointi ng_Icon_ 0040 30 30 31 5f 43 6f 75 72 73 65 5f 46 61 64 65 2e 001_Cour se_Fade. 0050 70 6e 67 23 23 2f 69 6d 61 67 65 73 2f 50 6f 69 png/im ages/Poi 0060 6e 74 69 6e 67 5f 49 63 6f 6e 5f 30 30 31 5f 46 nting_Ic on_001_F 0070 69 6e 65 5f 46 61 64 65 2e 70 6e 67 23 23 2f 69 ine_Fade .png/i 0080 6d 61 67 65 73 2f 50 6f 69 6e 74 69 6e 67 5f 49 mages/Po inting_I 0090 63 6f 6e 5f 30 30 31 5f 43 6f 6d 70 6c 65 74 65 con_001_ Complete 00a0 5f 46 61 64 65 2e 70 6e 67 23 23 2d 31 32 23 23 _Fade.pn g-12 00b0 31 30 25 23 23 30 2e 30 30 23 23 30 2e 30 30 23 10%0.0 00.00# 00c0 23 30 23 23 30 2e 30 30 30 30 30 30 23 23 30 2e #00.00 00000. 00d0 30 30 30 30 30 30 23 23 000000

Ach. The editor jumbled the hex and text columns of the wireshark report. Does an attached text file work better?

/media/uploads/RodColeman/mbed_query_forum_1.txt

I hacked HTTPClient.cpp in NetServices to make it work, if anyone else suffers from this problem.

The Server in question was fragmenting the TCP packets in response to HTTP GET, with the header (200 OK) in its own packet, and 174 bytes of payload in the next packet.

Line 519 of HTTPClient.cpp is a sscanf getting the 'Content-Length' from the headers - this was giving 0.

In an act of foul hackery, I removed the sscanf and made the returned m_DataLen fixed (178 bytes).

If anyone can point me where to look to fix it properly, I would be most grateful!

....next step:

HTTPClient.cpp get the size of the payload (m_DataLen) by having a

sscanf(m_respHeaders["Content-Length"].c_str() );

in other words: it goes through the HTTP headers looking for the argument "Content-Length".

The server device I am working with sends "Content-Type": text/html, but it does not send "Content-Length" in the header at all.

This fault in the server does not foul up Opera or Firefox browsers though, and although I got it working with a fudge, this lengthens the read time, since we must now wait for the server to be tired of being asked for bytes that are not there, and break the connexion.

Is there somewhere I can lodge a request to consider this for revisions of NetServices? I think it should fail gracefully (eg complain about the headers, or allocate 512 bytes etc), and this would make server faults easier to detect.

What do you think?