Re: Socket receive completely read

[Scott Klement <klemscot@xxxxxxxxxxxx>]

I think you've got the file data mixed up with the 'request chain' and
'response chain' ideas.

Normally when you connect to a server, you send HTTP keywords, one per
line, until your request is complete.   You are done with your request
when you send a blank line (a CRLF with no other data in front of it)
So, you could monitor for the end of a request chain by monitoring for
CRLF followed immediately by another CRLF.

After the request chain, the server will send back a 'response chain'.
The response chain follows the same format.   HTTP keywords, one per line,
followed by a blank line.   You can, once again, determine the end of
the response chain by looking for CRLF followed immediately by CRLF.

After the response chain will be the body of the actual data that is to be
sent.  This data is, however, NOT terminated in that same manner.  There
are two ways of telling when the data is done.   The first is the
'Connection: close' keyword.  If either the client or server sends this in
the request or response chain, then the server MUST close the socket after
all the data is sent.  Doing this, you can simply check to see when the
socket closes if the data was all sent.   Usually, there will also be a
'content-length' keyword that tells you how many bytes long the response
is, which is a good way to make sure you weren't simply interrupted by a
network problem.

Another method of encoding the data to ensure that it's all received is
the 'chunked' content transfer encoding.  This is used primarily for
persistent connections (connections that dont close after each request) or
for 'unknown lengths' of data... which might be the case with a servlet.
With chunked, a number is sent indicating the length of the data that
follows.  Then the data of that length in bytes is sent. This is called
a chunk.  You repeat that process until you get a chunk size of 0
indicating the end of the data.


If you want to see it laid out in code -- or don't want to mess around
with writing your own HTTP code to begin with -- you can check out my
open-source HTTPAPI service program here:

     http://www.scottklement.com/httpapi/


On Mon, 25 Nov 2002, Bartell, Aaron L. (TC) wrote:

> I have a socket RPG program that is sending a request to a Java Servlet on
> Tomcat.  The send works fine, but I am having trouble making sure I am
> getting the entire response; how do I know I have the whole response?
>
> Right now I stop reading after I get two CRLF's(0D 25 0D 25) right in a row.
> I then went out to the RFC 2616 only to find that I should probably be
> looking for "chunks".  Here is a quote from the site
> (http://www.ietf.org/rfc/rfc2616.txt).  Right now I am getting this as the
> last sequence of characters -> "0D 25 F1 0D 25 25 0D 25 F0 0D 25 0D 25"
>
> <snip>
>    The chunked encoding modifies the body of a message in order to
>    transfer it as a series of chunks, each with its own size indicator,
>    followed by an OPTIONAL trailer containing entity-header fields. This
>    allows dynamically produced content to be transferred along with the
>    information necessary for the recipient to verify that it has
>    received the full message.
>
>        Chunked-Body   = *chunk
>                         last-chunk
>                         trailer
>                         CRLF
>
>        chunk          = chunk-size [ chunk-extension ] CRLF
>                         chunk-data CRLF
>        chunk-size     = 1*HEX
>        last-chunk     = 1*("0") [ chunk-extension ] CRLF
>
>        chunk-extension= *( ";" chunk-ext-name [ "=" chunk-ext-val ] )
>        chunk-ext-name = token
>        chunk-ext-val  = token | quoted-string
>        chunk-data     = chunk-size(OCTET)
>        trailer        = *(entity-header CRLF)
>
>    The chunk-size field is a string of hex digits indicating the size of
>    the chunk. The chunked encoding is ended by any chunk whose size is
>    zero, followed by the trailer, which is terminated by an empty line.
> </snip>
>
> My question is does anybody have something like this already laid out in
> code so I don't have to cover all of the different bases that a host could
> possibly send back to me?  Or does somebody have a catch all that will allow
> me to always make sure that I have received the entire response?  Or is the
> sequence that I have above ("0D 25 F1 0D 25 25 0D 25 F0 0D 25 0D 25") what I
> need to be looking for to know that the request has completed in its
> entirety?
>
> Thanks,
> Aaron Bartell