Fix Arduino Emulator CI to produce an executable with https://github.com/MitchBradley/PosixAsyncTCP

[mathieucarbou]

This PR is a follow up of the previous one #420 to add support for Arduino Emulator.

lwip inclusion should be added only if lwip is there or when HOST is not set.

[mathieucarbou]

@MitchBradley : FYI.

I updated the CI workflow to compile against your AsyncTCP posix version instead of the one from our repo using lwip and FreeRTOS functions.

I can produce an executable, which, when launched, will listen to port 8080.

But a curl -v http://127.0.0.1:8080/ leads to nothing FYI.

[MitchBradley]

Great, now that this piece is stable, I can proceed to the next step. I will figure out why curl does nothing with your test case, then work on a focused test case for the close state machine problem. In my full FluidNC build, everything seems to work - serving web pages, interacting via a REST API, websockets, and WebDAV, and simultaneously simulating a CNC machine in a web app that is connecting back on a websocket.

[MitchBradley]

The problem with the example is that it needs

  #include <AsyncTCP.h>
  ...
  setup() {
    PosixAsyncTCPManager::getInstance().begin();

There is probably a "more compatible" way; Arduino-Emulator has an Ethernet class and a WiFi class mock on top of it. I need to study the other AsyncTCP implementations to see how they are initialized.

[mathieucarbou]

The problem with the example is that it needs

  #include <AsyncTCP.h>
  ...
  setup() {
    PosixAsyncTCPManager::getInstance().begin();

There is probably a "more compatible" way; Arduino-Emulator has an Ethernet class and a WiFi class mock on top of it. I need to study the other AsyncTCP implementations to see how they are initialized.

I confirm, it works perfectly! I will push the fix, thanks!

[MitchBradley]

Actually, I think I can do better. I think that the call should be embedded in PosixAsyncTCP > AsyncServer.begin().

[mathieucarbou]

Actually, I think I can do better. I think that the call should be embedded in PosixAsyncTCP > AsyncServer.begin().

👍

Quick question also regarding AsyncTCP... I tried to compile the AsyncTCP we have (based on lwip). The only issue I had was all the FreeRTOS usage we do in AsyncTCP, but I was able to git clone and compile lwip repo.

Do you think that if we were using std::thread and another queue system (and semaphore) the library could be reused as-is for the emulator ?

Because if yes, then maybe we could try to refactor AsyncTCP to support the HOST macro ?

What was the driven factor for the reimplementation of AsyncTCP ?

[MitchBradley]

PosixAsyncTCP now starts the polling thread automatically in AsyncServer::begin(), so the explicit call in setup() is unnecessary, albeit harmless.

Re the reimplementation - it seemed like the easiest path given the fact that every host system implements ultra-stable BSD networking out of the box. lwip requires a lot of configuration and I imagined spending inordinate amounts of time fighting it - and it would still be sitting on top of the host's BSD networking, so an extra layer of complexity, configuration, and dependency management for no benefit. Copilot, backended by ChatGPT and Claude, did most of the heavy lifting for the implementation.

I have a FreeRTOS mock that I use in FluidNC, implementing queue, semphr, task, and timers on top of C++ mutexes, semaphores, threads, and chrono. I only implemented what my app needs. Right now it is embedded in the FluidNC code base, not a published package.

[MitchBradley]

One FreeRTOS feature that I probably cannot replace is queues, since I need the ISR-safe versions and C++ doesn't have that. Maybe I could use the Embedded Template Library but that would be yet another framework to learn to use.

[mathieucarbou]

That’s an impressive work!

[mathieucarbou]

And did you considered using AsyncTCPSock ? This one is bsd socket based also.

[MitchBradley]

I did not know of the existence of AsyncTCPSock. Had I known, I probably would have used it. I am looking at it now.

[MitchBradley]

Ah, I see what is happening. AsyncTCPSock sits on top of lwip, and depends on FreeRTOS and has some ifdef-guarded ESP dependencies. I think its primary difference from the other AsyncTCP is its use of lwip's BSD socket layer instead of the netconn interface or whatever that is called. So my concerns about adding an unnecessary lwip layer on top of an existing TCP hold for it too.

Thanks for pointing it out.