Re: Weird Sockets Issue...

[Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>]

I'm a bit confused by this message.  I know you've written sockets 
programs before (GETURI for example) so I don't understand why you're 
describing the normal behavior of a TCP socket and referring to it as 
"weird".

This makes me think that I don't understand your question correctly. 
Because what you describe is the normal behavior of the TCP protocol. The 
TCP protocol does not think of your data as a fixed-length record, it 
thinks of it as a continuous stream.

When (for example) 3000 bytes are written in one end, they're broken up 
and sent over the network according to an algorithm that's supposed to 
provide the best throughput.  When your program receives the data, it 
might get all 3000 bytes at once.  Or it might get 500 bytes, then 1200 
bytes, then 800 bytes, then 500 bytes in 4 separate calls to recv().

In debug, you'd be more likely to get all 3000 at once because your 
program is running slower, so the network has more time to deliver more 
data.

Since there are a lot of variables involved in determining how the data is 
broken up, you should never rely on data arriving in any particular chunk 
size.  Always write your applications so that they keep receiving data in 
a loop until something signals the end of the data.

And that's how you know not to call recv() or read() again -- when you've 
been told that you've reached the end of the data.  Otherwise, you'll 
have the "hanging" effect (more correctly called "blocking") that 
you're describing.

In many protocols, the data will be broken up into "lines".  A line will 
end with CRLF characters.  Therefore, to make sure you've received the 
whole line, you keep receiving until you have the CRLF characters.

In other cases a length will be sent that tells you how long the data is. 
For example, the "content-length" sent in an HTTP response. Then you can 
write a loop that receives exactly that many bytes, and it knows it's done 
when that many have been received.  Another example of this is the 
"chunked" encoding used in HTTP.

In FTP, when the data is complete, a message is sent to the control 
channel to indicate the end of the data. (It also closes the socket on the 
data connection).

In SMTP or NNTP a line of data ending in CRLF and consisting of only the 
"." character is used to indicate the end of the data.

Naturally, you can dream up any method you want of signalling the end of 
the data if you're designing your own protocol.

Does that answer your question, or am I misinterpreting it?

---
Scott Klement  http://www.scottklement.com



On Wed, 1 Feb 2006, Brad Stone wrote:

> Currently playing around with some client sockets
> programming and having a weird issue I never experienced
> before.
>
> When I run my application one of the response I get back
> from the server isn't always complete.
>
> The odd thing is, when I use debug to debug the app, it is
> always complete.
>
> I shut off debug, and it doesn't receive the entire
> response, which technically would require another read from
> the socket to get the rest of the data (actually maybe more
> than 1 read).
>
> But, I can't just add another read, because if it DOES get
> all the data, it hangs on the 2nd read.
>
> I'm using select, but on reads when I get all the data the
> first time, the select gives me a timeout error.