Suggested improvement to Arduino/libraries/ESP8266WebServer/src/detail/RequestHandlersImpl.h

[elfalko]

Basic Infos

request Handler for .gz files works wrong if specified file path is to same file without .gz ending.

Hardware

Module: Generic ESP8266 Module

Description

I suggest wrapping the part looking for a gzipped version of a requested path in the same part that looks for the index.htm if the given path isn't found.
See sketch below

Problem description

eg. specified file path for "/web/index.html" file while /web/index.html.gz file exists
Expected behaviour:

1. StaticRequestHandler::handle(...) should set content type to "txt/html"
2. realize there is no such file
3. try to attach ".gz" to path, try to find file again, serve if ok (with Content-Encoding:gzip Header
4. if still not found, attach index.htm and try to find that

Real behaviour

1. StaticRequestHandler::handle(...) should set content type to "txt/html"
2. realize there is no such file
3. try to attach index.htm to path, try to find file, serve if ok
4. since that(in my case "/web/index.htmlindexhtm") wasn't found either, it attaches ".gz" and tries to find that
5. /web/index.htmlindex.htm.gz of course isn't found and nothing happens

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 4MB
CPU Frequency: ?80Mhz?
Flash Mode: ?qio?
Flash Frequency: ?40Mhz?
Upload Using: SERIAL
Reset Method: nodemcu

Sketch

Current code (in Arduino/libraries/ESP8266WebServer/src/detail/RequestHandlersImpl.h):

class StaticRequestHandler : public RequestHandler {

    {...}

    if (!_isFile) {
        // Base URI doesn't point to a file.
        // If a directory is requested, look for index file.
        if (requestUri.endsWith("/")) requestUri += "index.htm";

        // Append whatever follows this URI in request to get the file path.
        path += requestUri.substring(_baseUriLength);
    }
    DEBUGV("StaticRequestHandler::handle: path=%s, isFile=%d\r\n", path.c_str(), _isFile);

    String contentType = getContentType(path);

    // look for gz file, only if the original specified path is not a gz.  So part only works to send gzip via content encoding when a non compressed is asked for
    // if you point the the path to gzip you will serve the gzip as content type "application/x-gzip", not text or javascript etc...
    if (!path.endsWith(".gz") && !_fs.exists(path))  {
        String pathWithGz = path + ".gz";
        if(_fs.exists(pathWithGz))
            path += ".gz";
    }

{...}

};

Suggestion:

if (!_isFile) {
// look for gz file, only if the original specified path is not a gz. So part only works to send gzip via content encoding when a non compressed is asked for
// if you point the the path to gzip you will serve the gzip as content type "application/x-gzip", not text or javascript etc...
if (!path.endsWith(".gz")) { //no !_fs.exists(path) needed since that is done by !_isFile
String pathWithGz = path + ".gz";
DEBUGV("StaticRequestHandler::handle: Looking for .gz file at %s\r\n", pathWithGz.c_str());
if (_fs.exists(pathWithGz)) {
path += ".gz";
DEBUGV("StaticRequestHandler::handle: .gz file found.\n");
} else {
DEBUGV("StaticRequestHandler::handle: NO .gz file found, checking for index.htm\n");

      // Base URI doesn't point to a file.
      // If a directory is requested, look for index file.
      if (requestUri.endsWith("/")) requestUri += "index.htm";
      DEBUGV("expanding path\n");
      // Append whatever follows this URI in request to get the file path.
      path += requestUri.substring(_baseUriLength);
    }
  }
}

[elfalko]

While we're at it, this does not serve included image files the way it should if there is compressed serving involved.

Example:
in a given index.html file a logo is included with the standard stuff
<img src="logo.png" alt="MyCompanyLogo"/>

I now serve the index.html path. The (modified) handler finds the index.html.gz, attaches the appropriate
Content-Encoding: "gzip" HTTP header and serves it. Everything ok till here

Now it serves the logo. It returns a Code 200 since it finds it, but returns it with the Content-Encoding: "gzip" header and the Content-Type: text/htmlas well. This results in the image not loading.

Same goes for favicons. Both should not be prezipped since the compression does not really help with most image formats.

Response Header index.html:
Access-Control-Allow-Origin:*
Cache-Control:max-age=86400
Connection:close
Content-Encoding:gzip
Content-Length:1001
Content-Type:text/html

Response Header logo.png/favicon.ico:
Access-Control-Allow-Origin:*
Cache-Control:max-age=86400
Content-Encoding:gzip
Content-Length:1001
Content-Type:text/html

EDIT: fixed escaped img include sequence for proper display

[devyte]

@elfalko could you please prepare a PR with your suggestions?

[elfalko]

It might take some time till i get around to that, since i have since switched to the esp32 and got more pressing issues, sorry.

[earlephilhower]

I think the original bug was filed against a very old core version. Testing the simple example works. (i.e. have index.html.gz in FS, but user requests "index.html", the request is served from index.html.gz silently w/no errors).

The code flow described also does not happen. The addition of "index.htm" is only attempted if !isFile() and the request ends in "/".

Closing as can't repro. If someone finds another case, please open a new issue w/MCVE.