Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 13 total
Thread Dirty Server Request's ResponseHeaders
Thu, Jan 3 2019 12:16 PMPermanent Link

ooptimum

While working on my project, I noticed that IE and Firefox/Chrome process server responses differently: in IE, they are interpreted as text/plain, which is wrong, and in Firefox and in Chrome, they are treated as application/json, as it should be. I began to deal with this and found that after the server request is executed in IE, Request.ResponseHeaders.Text contains string starting with the LineFeed symbol:
IntToStr(Ord(Request.ResponseHeaders.Text[1])) = 10.

I noticed that for some reason a single header field is returned in Request.ResponseHeaders - the Content-Type field. In IE, its name is not the same as HTTP_HEADER_CONTENTTYPE:
SameText(Request.ResponseHeaders.Names[0], HTTP_HEADER_CONTENTTYPE) = False, since
IntToStr(Length(Request.ResponseHeaders.Names[0])) + ' ' + IntToStr(Length(HTTP_HEADER_CONTENTTYPE))) = "13 12", and
IntToStr(Ord(Request.ResponseHeaders.Names[0][1])) = 10.

As I said before, when the same request is made in Firefox or in Chrome, everything works properly.

I do not understand why this is happening, since inside TServerRequest.Complete all header lines are trimmed, and I do not see any errors inside Trim function. Can anybody confirm this error?
Fri, Jan 4 2019 11:36 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

<< I do not understand why this is happening, since inside TServerRequest.Complete all header lines are trimmed, and I do not see any errors inside Trim function. Can anybody confirm this error? >>

What are you using to generate the response ?  Are you using the EWB Web Server ?  And, is it a dataset response or something else ?

Tim Young
Elevate Software
www.elevatesoft.com
Sat, Jan 5 2019 9:19 PMPermanent Link

ooptimum

<< What are you using to generate the response ?  Are you using the EWB Web Server ?  And, is it a dataset response or something else ? >>

Actually the response is generated by the Express framework on a Node.js server, but it doesn't matter. You can request any third party server to get this behavior, as I did in the attached example.



Attachments: test.zip
Sun, Jan 6 2019 12:07 AMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 1/5/2019 9:19 PM, ooptimum wrote:
> Actually the response is generated by the Express framework on a Node.js server, but it doesn't matter. You can request any third party server to get this behavior, as I did in the attached example.

I'm not really seeing what you described in the original post - i get
back multiple headers (using either IE, Edge or Chrome) and content type
is present OK.

Other general notes :

Servers can and do customize response headers back to the browser (based
on User-Agent in request headers) so IE and Chrome can often get
different headers back.

If you override User-Agent with custom one then sometimes you can get
back weird results sometimes - not sure if your app is doing that.

For your sample the IE is getting back less headers from server than
Edge/Chrome (some of it due to IE not getting gzip content and not using
keep-alive etc).


Finally in EWB i'm seeing lot less headers than returned (as per browser
debug view). I suspect this is browser design but would be good to get
Tim to confirm.

App basically sees 3 headers (Content-Type, Expires and Last-Modified).

Chrome shows 11 headers returned - most of them are related to
connection, caching, CORS and content encoding which should be
transparent to the app so it makes sense those are not visible.

Some of the others - like custom ones - i'm surprised don't show up though.

This is a sample set of headers from browser

   Access-Control-Allow-Origin: *
   Cache-Control: max-age=2592000
   CF-Cache-Status: HIT
   CF-RAY: 453d592d37713f9b-YUL
   Connection: keep-alive
   Content-Encoding: gzip
   Content-Type: image/svg+xml
   Date: Sun, 06 Jan 2019 04:29:21 GMT
   ETag: W/"5b3ddd2d-6c3"
   Expires: Tue, 05 Feb 2019 04:29:21 GMT
   Last-Modified: Thu, 05 Jul 2018 08:56:13 GMT
   Link: <http://framework7.io/i/vi-logo.svg>; rel="canonical"
   Server: keycdn-engine
   Transfer-Encoding: chunked
   Vary: Accept-Encoding
   X-Cache: HIT
   X-Edge-Location: camo


and in app it's basically

   Content-Type: image/svg+xml
   Expires: Tue, 05 Feb 2019 04:29:21 GMT
   Last-Modified: Thu, 05 Jul 2018 08:56:13 GMT


Raul
Sun, Jan 6 2019 6:13 AMPermanent Link

ooptimum

Raul,

Thank you for detailed explanation of http headers, but it's quite fruitless for me, as I know almost everything about this topic.

<< Finally in EWB i'm seeing lot less headers than returned (as per browser
debug view). I suspect this is browser design but would be good to get
Tim to confirm.

App basically sees 3 headers (Content-Type, Expires and Last-Modified). >>

My problem is that when I run my app within the IDE or directly in IE, the Content-Type clause is polluted with extra Line-Feed in the begginning, so ResponseContentType property of the corresponding request is also empty.

My environment is Windows 8.1 64-bit and IE 11.

In the attachment is a screenshot of the Output windows of the example app.



Attachments: output.png
Sun, Jan 6 2019 10:49 PMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 1/6/2019 6:13 AM, ooptimum wrote:
> My problem is that when I run my app within the IDE or directly in IE, the Content-Type clause is polluted with extra Line-Feed in the begginning, so ResponseContentType property of the corresponding request is also empty.
>
> My environment is Windows 8.1 64-bit and IE 11.
>

No problem - my main point was simply that i'm not seeing it here with
your app and IE.

However I'm on Win 10 and IE 11.371.16299.0 and all is OK.

So i dug up an older Win 8.1 VM i have that has IE 11.0.9600.19204.
Update Versions: 11.0.100 - this appears to be latest as per windows update.

I'm seeing same behavior there - the response headers
(getAllResponseHeaders call) that browser returns appears to have a
leading LF (10 decimal) like you described in old IE 11.

Looks like the browser did not properly strip out the full CRLF between
this and previous header and left the LF there.

Anyways the Request.ResponseHeaders.Text basically looks like this on
Win 8 (note leading LF/[10]) :
[10]Content-Type:image/svg+xml[13][10]Expires:Tue, 05 Feb 2019 04:26:11
GMT[13][10]Last-Modified:Thu, 05 Jul 2018 08:56:13 GMT

Short answer appears to be that IE11 in Win 8 is broken and Microsoft
was aware of this since newer IE11 in Win10 fixes this issue.

EWB fix might be as simple as modifying the webhttp unit and changing
the TServerRequest.Complete function where
FResponseHeaders.Text:=FHttpRequest.getAllResponseHeaders is assigned to
check for leading LF (10) and strip it from string before assignment.

Tim would have weigh in on this one as to the best way to solve in EWB.

Raul
Mon, Jan 7 2019 1:10 AMPermanent Link

ooptimum

Raul wrote:

<< Short answer appears to be that IE11 in Win 8 is broken and Microsoft
was aware of this since newer IE11 in Win10 fixes this issue. >>

Exact same problem appears in Windows 7 too. It's still 2nd popular desktop Windows OS, according to research on http://gs.statcounter.com/windows-version-market-share/desktop/worldwide, so it shouldn't be neglected, even thought IE isn't supported anymore nor has any popularity.

<< Looks like the browser did not properly strip out the full CRLF between
this and previous header and left the LF there. >>

There are these lines in TServerRequest.Complete method just to address such problems (lines 357-359 of webhttp.wbs):

        { Clean up response header values in case they contain leading spaces }
        for I:=0 to FResponseHeaders.Count-1 do
           FResponseHeaders.ValueFromIndex[I]:=Trim(FResponseHeaders.ValueFromIndex[I]);

The question is why it's still not sufficient and we see this leading LF.
Mon, Jan 7 2019 9:08 AMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 1/7/2019 1:10 AM, ooptimum wrote:
> Exact same problem appears in Windows 7 too. It's still 2nd popular desktop Windows OS, according to research on http://gs.statcounter.com/windows-version-market-share/desktop/worldwide, so it shouldn't be neglected, even thought IE isn't supported anymore nor has any popularity.

That's something you'd need to take up with Microsoft - it's their product.

EWB can work around this now that problem is known but root cause is
IE11 lacking latest updates from MS

> There are these lines in TServerRequest.Complete method just to address such problems (lines 357-359 of webhttp.wbs):
>
>           { Clean up response header values in case they contain leading spaces }
>           for I:=0 to FResponseHeaders.Count-1 do
>              FResponseHeaders.ValueFromIndex[I]:=Trim(FResponseHeaders.ValueFromIndex[I]);
>
> The question is why it's still not sufficient and we see this leading LF.

The issue is not at value level so fix would have to be something like
this to clean up source headers string - this might be too simplistic
but bascially replace:

FResponseHeaders.Text:=FHttpRequest.getAllResponseHeaders;

with

if (FHttpRequest.getAllResponseHeaders<>'') AND
(FHttpRequest.getAllResponseHeaders[1]=#10) then

FResponseHeaders.Text:=copy(FHttpRequest.getAllResponseHeaders,2,length(FHttpRequest.getAllResponseHeaders)-1)
else
  FResponseHeaders.Text:=FHttpRequest.getAllResponseHeaders;

Raul
Mon, Jan 7 2019 10:47 AMPermanent Link

ooptimum

Raul wrote:

<< That's something you'd need to take up with Microsoft - it's their product.

EWB can work around this now that problem is known but root cause is
IE11 lacking latest updates from MS >>

I agree with you in that it's IE fault, because it's IE returning LF in the beginning of headers string. And I am by no means attacking EWB, especially since the problem isn't really a big deal. But it amazes me that in all these years nobody has discovered this problem, or it has appeared recently.

<< The issue is not at value level so fix would have to be something like this to clean up source headers string >>

I am not saying that your method is wrong or will not work. Of course you can sanitize the whole headers string before it is handled further. There is no any doubt in that. I am trying to say that those lines that already exist in the code should have cleared everything in the beginning of headers after splitting them by lines, but for some reason this does not work. It you will check the log output, you will notice there is "[10]Content-Type" header among others. This LF in the beginning should be cleared by Trim function, which is called on each header in the list, but it's not cleared for some unknown (yet) reason.
Mon, Jan 7 2019 10:58 AMPermanent Link

ooptimum

ooptimum wrote:

<< This LF in the beginning should be cleared by Trim function, which is called on each header in the list, but it's not cleared for some unknown (yet) reason. >>

OMG, I must have not quite recovered after a severe flu, only by this I can explain the fact that I did not notice that Trim applies only to the values of the headers, not to their names.
Page 1 of 2Next Page
Jump to Page:  1 2
Image