Compilation errors in Arduino 1.8.8

[ace42588]

I keep running into compilation issues when verifying the example sketch that don't make much sense. I didn't capture the compiler output from the first couple issues, but I'll paraphrase

• request = requestDoc.to(); -Rest-Esp8266.h - compiler reported not enough arguments provided to DynamicJsonDocument. Resolved by blowing away the library as downloaded by Arduino and replaced with a pull from GitHub

• using Exception = Exception; - Node.h:85:42: error: declaration of 'using Exception = class Rest::Exception<Rest::Node >'. Can be resolved by replacing the alias in Node.h

• typedef Esp8266RestRequestHandler RestRequestHandler; - Restfully.h:7:1: error: expected constructor, destructor, or type conversion before 'typedef'. Latest error

Seems to me that I'm missing something or my library keeps getting corrupted somehow. Is there anything unique needed to compile the example sketch? I am using ArduinoJson 6.5 and included stdio.h just for chuckles

[guru-florida]

Hi @ace42588. As noted in part of the doc only the ArduinoJson 6.x is supported at the moment, I think that is what may be happening here. ArduinoJson had a significant interface change betwee v5 and 6.

Honestly, I would hold off from using this project for a few weeks. I am in the middle of some changes at the moment. Mostly, the use of .on(, GET(func), PUT(func)) was causing all sorts of C++ issues and cryptic error messages if function signatures weren't perfect.

For ease of use, I've changed to .on().GET(func).PUT(func) kind of syntax. In the new format the on() returns a node reference and then the .GET() and .PUT(), etc, are just chained function calling. The compiler no longer has to deduce arguments in a pseudo-magic way so even if you make an error in your callback types you'll get a more sensical error to point you in the right direction.

I am almost done the work. I am using this project in another Esp8266 project but I would love to have another tester. Are you interested? If so, can you provide me more details on what hardware and libraries (+versions) you are using? The core of the Restfully code isnt tied to any particular Json or network lib, but has a single file (Rest-Esp8266.h) that ties specific dependencies to the Rest core....so once I get my latest changes checked in I can start adding the compatibility including both ArduinoJSON APIs (<v5 and v6 onward) and non-8266 devices.

[ace42588]

I'd be happy to help test.

• Hardware: generic ESP-01
• Arduino v1.8.8
• Esp8266 core v2.5.0
• ArduinoJson v6.9 (also tried 6.5)

[brunokc]

I'm available to test as well on PlatformIO (using Arduino framework) on a ESP32 -- I know you said you're not interested at this point, but I am :) I'd like to see if I can make it work there. Right now, I'm applying all sorts of hacks to make it compile.

I'm on:

• PlatformIO Core 3.6.5
• ArduinoJson 6.9.1
• ExpressIf 32 1.7

Thanks.

[brunokc]

What do you know, I got it working on ESP32. Not a lot of changes needed, mostly did this:

• Dropped paged-pool.*
• Now using the general WebServer instead of specific ESP8266WebServer one
• Fixed Rest-Esp8266.h:
  • Fixed DynamicJsonDocument constructor, now specify some value for initial allocations
  • Fixed RequestHandler::handle() which was ignoring the return value of the handler
• Missing #include <stdio.h> and <string.h> (using string.h for memset() instead of memory.h as it was originally -- for some reason memory.h is not available in my environment)

Based on what I see here, you seem to have fixed some of these already and went beyond that, so I don't think it's worth posting a PR. Let me know if you think otherwise though.

Looking forward to another release (and push to PlatformIO).

[guru-florida]

@brunokc Yes, paged-pool is unused at the moment but possibly used in the future. I would like to be able to see the diff on the changes you did, are you working from a fork? Glad to see it wasnt hard to get working on ESP32!

[guru-florida]

Good news guys! I've completed my changes and added some great features. The hard part is over, I just have to update the Arduino examples and I can commit. (I estimate a few days max)

• Can join different Endpoint collections.
  • Example: Have a main that branches off to multiple "sub-systems".
• Now supports class methods as handlers.
  • Works great with first feature. Have a parent collection with static handlers, then attach instance handlers associated with a 'this' object.
  • Also, the instance object can instead be a callback function that returns the instance object. For example, a rest endpoint like /devices/<device-id>/control/start might use the device-id parameter to lookup and return the right device object instance to invoke the "start" endpoint handler.
• (Mostly) less cryptic error messages
  • though I wish it was still better but the old ones might as well have been machine code.

Breaking Changes

• As mentioned, all the endpoints change from endpoints.on("/path/etc", GET(func), PUT(func)) to endpoints.on("/path/etc").GET(func).PUT(func).
• Endpoints class is just a collection of path nodes. The Node class has all the power with methods on(), resolve(), with(), GET, PUT, PATCH, etc. This shouldnt be a breaking change in most cases other than the first point.

Joining Endpoint collections

As mentioned in features, we can join two Endpoints collections using the .with(args) method. The with method will return the right-hand Endpoints collection so you can keep using .on(), .GET(), .PUT(), etc in a method invoke chain. I am using Restfully in another project (Nimble) where the Rest interface is key to interfacing with the internal sensor devices, etc. in a plug-and-play way so splitting the Rest endpoints into multiple units was very important.

Typically, you would use with() something like this:
Endpoints ep1, ep2; // this type is actually system dependent

ep1.on("/api/sys").with(ep2).on("status").GET(status_func);

or nicely indented to show endpoint level:

ep1
    .on("/api/sys")
    .with(ep2)
        .on("status")
            .GET(status_func)
        .on("statistics")
            .GET(statistics_func)

In the above example the status and statistics endpoints are actually defined in the ep2 collection, then attached to the /api/sys node of the ep1 collection. The with() method always returns the right-hand Endpoints collection.

In the above examples both collections have the same handler type (static functions). We can also change the code to use method instances for ep2:

class Sys {  status(); statistics() }
Sys mysys;

Endpoints ep1;     // this type is actually system dependent
Endpoints<Sys> ep2;
ep1
    .on("/api/sys")
    .with(mysys, ep2)
        .on("status")
            .GET(&Sys::status)
        .on("statistics")
            .GET(&Sys::statistics)

Actually, ep2 is optional here since we never actually use ep2 outside the chain. The Endpoints class can construct it as an anonymous type and store it within the parent collection.

class Sys {  status(); statistics() }
Sys mysys;

Endpoints ep1;     // this type is actually system dependent
ep1
    .on("/api/sys")
    .with(mysys)      // this creates a new anonymous collection with Sys class and returns it
        .on("status")
            .GET(&Sys::status)
        .on("statistics")
            .GET(&Sys::statistics)

The with(...) method has overloads that support these args:

• <Endpoints> - join Endpoints at the current node. (both left and right Endpoints collections must be the same type.)
• <object>, <Endpoints> - join Endpoints at current node, but use object as the 'this' pointer.
• <object> - construct an anonymous Endpoints collection and store within the parent collection. The object is still used as the 'this' pointer during invokes.
• <callback_function>, <Endpoints> - join Endpoints at current node. At Request time, the callback_function is invoked to lookup and return the object reference. callback_function prototype would be Klass cb_func(Rest::UriRequest& r) where Klass is your object type and UriRequest contains the URL and parameters parsed so far and your code may use a parameter as the key to lookup your object instance dynamically.

Class Method Handlers vs Static Handlers

In all cases, when you use with(...) to attach an object instance to a member function the this object is bound to the member function at Request time to produce a statically invokable function call (think std::bind). So in the above examples resolving from the parent (containing static handlers) will always return a static function handler regardless of joined instance member based Endpoints.

You can also create stand-alone Endpoints that return class-method handlers as well but all method handlers in an Endpoints collection must be from the same class type. You would resolve a URL to get a class-method handler, and would perform the invoke yourself using the .* operator.

[brunokc]

@brunokc Yes, paged-pool is unused at the moment but possibly used in the future. I would like to be able to see the diff on the changes you did, are you working from a fork? Glad to see it wasnt hard to get working on ESP32!

I don't have a fork, I just cloned, checked out the 0.2.0-beta release and applied my changes on top on a local topic branch. I can post a PR from that branch, if you'd like. I can do that tonight when I get home.

[guru-florida]

@brunokc I've added you as a collaborator, so you can create a branch and post changes there. Thanks!

[brunokc]

@guru-florida , this update looks awesome! Looking forward to giving it a try.

[guru-florida]

I've checked in the changes to master.

@ace42588 Can you take a crack at it? Are you running from a Git clone or straight from Arduino lib manager (if so, then I have to do a release first and wait a day for Arduino to catch the new release)

@brunokc I've taken in your edits manually (without merging PR since the code has changed so much). The library should work with with both Esp8266 and Esp32 now. There are two files in src/Platforms/Esp8266.h and src/Platforms/Esp32.h that define the right classes and I defined the Esp32.h based on your PR diffs. I cant test it here but I bet you can figure out any mistake I made if I did :) . It should look pretty simple I think. Let me know one way or the other.

Example SimpleRestServer is updated as well.

[guru-florida]

I've updated the main README to document required changes and new features

[brunokc]

Thanks for the update, @guru-florida. Here's a summary of the things I'm seeing:

1. For some reason PlatformIO likes to compile everything under a folder (I just started using it for this project, so I may be missing something). Because of this I'm hitting some of the same problems I had to fix for 0.2.0-beta. Namely, I had to rename or remove files that didn't apply to my case (e.g.: import/paged-pool.*) so things would compile.
2. As before, I can't find memory.h, switched to string.h. Not sure how to solve this one in a general way.
3. Platforms/generics.h still had a reference to ESP8266WebServer.h. I removed it.
4. Platforms/generics.h seems to be platform agnostic. Does it really belong under the Platforms folder?
5. I have to admit having to use arrows now on RestRequestHandler (e.g.: restHandler->on(...)) threw me off, knowing that I didn't allocate it on the heap, but as a global. Is that the intention going forward? If so, I believe README.md needs to be updated to reflect that.
6. I was getting some compilation warnings due to member initialization ordering in Rest::Generics::Request's first constructor (generics.h). By moving the initialization of server to happen after the invocation of the base class' constructor (TUriRequestFragment), the warning went away.

My code is building now. I'll run some basic tests and let you know.

Thanks!

[guru-florida]

@brunokc I've found an ESP32 (Wemo+LCD) so I am going through updates to support both ESP8266 and ESP32. Your list is really helping!! I installed PlatformIO and gave it a shot but that's all new to me so ran into walls....I will definitely try again as the VSCode and PlatformIO plugin looks like a great pair.
(1) I did remove the "import" folder with the paged and pool files.
(2) I havent run into this yet on both Windows and Mac builds. Very strange.
(3) Fixed.
(4) I think so. The generics there are to assist in building platform configs. They tie a WebServer, a Json library and the Restfully core into a single (more friendly) solution. Those generics are not required for the Restfully core and you could use the Restfully core without it...like I do on Linux/Mac gcc/clang builds outside Arduino.
(5) I agree with you actually. I did this so I didnt have to create a bunch of inline functions that just delegate to the Endpoints class. Overriding the -> just allowed a simple way to go straight to the Endpoints class. There are only 2 or 3 methods though so I should just do this and go back to dotted. Arduino guidelines say -> is not Arduino friendly. You got me on the README not updating -> usage. hehe
(6) I get these warnings too, a few signed/unsiged and initialization order. I will post a fix soon.

I updated the Restfully example to support ESP32 with minimal #define tests (i.e. 8266 or ESP32?). I am currently debugging why on ESP32 the LED endpoints arent working though, returning 404.

[guru-florida]

Ok, fixed the 404 issue. I have the example working well on both Esp8266 and ESP32 devices!

(5) Also, you dont need to use the -> operator anymore. On closer look it is only the on(...) method needed on the RestHandler class and from the first on() all the other methods are accessible on the Node class. So a single inline forwarding method did the trick.

(2) I still havent run into the memory.h / string.h issue. Can you tell me exactly where that occured?

(6) I fixed the initialization order. This is an error on ESP32 and warning on 8266.

Can you try out the latest asap? I would like to generate a release so Arduino Library Manager picks up the new version....the old version is pretty bad at this point. :) I am concerned about the memory/string.h though.