Conversation
|
/benchmark -f ngx-php |
|
👋 |
|
🚀 Benchmark run triggered for |
Benchmark ResultsFramework:
Full log |
|
Can you change this to tuned? |
|
Why ? |
|
If you explain why we need to change it, we can decide about any change that is breaking the production, to delete it or use tuned. |
|
worker_cpu_affinity auto; These are not default nginx, can you find anywhere in nginx docs recommending to set these as default configs? |
|
In the nginx documentation https://nginx.org/en/docs/. Core functionality https://nginx.org/en/docs/ngx_core_module.html
Ngx-php https://github.com/rryqszq4/ngx-php#directives I don't understand anything. |
|
It's like node.js if they can add reuseport or cluster. As default don't work with this config. Or like asp.net configuring the kestrel. |
|
worker_cpu_affinity auto these seem to be clearly tuned configs, they are documented in the docs but not typically used on a prod deploy, it should be nginx handling these not us setting deliberately. nodejs doesnt work without reuseport, it is a core config, nginx does work fine without it and is the default config to not set it. Those aspnet configs are to be removed from any prod type |
|
Nginx
Later:
Nodejs Etc, etc,.. |
|
About that config that is not "production". Well, marked as "production": https://github.com/MDA2AV/HttpArena/blob/main/frameworks/aspnet-minimal_nginx/proxy/nginx.conf#L1-L6 .... |
|
What is production for me actually: Production
Tuned
This bench seems to biased to asp.net, if the framework don't use the same automatic config then is tuned.
We are developers, we create code, and the asp.net opinionated classes are not always the optimal or more performant. If any dev want easy automatic solutions, they can check the code in this repo and choose. |
|
Yea these nginx flavour "frameworks" are going to be a pain in the ass, there are a trillion ways to configure nginx and every can claim they are production because there are use cases for all those configs. This is more of a nginx module than a framework which sort of invalidates prod/tuned logic applied to it because nginx is not a web framework. These prod rules are targeted to frameworks like Spring, Quarkus, Helidon, Elysia, ASP.NET, Laravel, FastHttp, Gin, etc which are a different category, for this reason we need a new category for anything that falls on reverse proxies, web servers like caddy, nginx, traefik, etc So I'll just move whatever is nginx based to this new category, apples to apples |
|
Like you want. Thank you. |
|
We can't move ngx-php to the gateway test, as minimum need 2 docker containers !! |
|
So you think it is fair to have ngx-php and laravel under same category? |
|
Any framework than use |
|
Because it does not provide the same functionality. Oh cmon you know exactly what I am talking about, so anyone can just add a module to nginx or some low level engine to support basic features like routing and request parsing and compete with a full feature web framework? |
|
Later I will add Symfony and Laravel to this bench, but not with Even Symfony and Laravel are always slow, is the same which platform they use. |
|
If you have a Rust framework, you have all the possibilities to configure in the best way to be faster than any module from nginx, than later create all the code in an interpreted language, not compiled !! |
|
But you have to code in Rust to use it so it sucks already and nobody cares Build a basic web api like we enforce here is doable fast in any language, if you have to build a large reliable and mantainable one nobody is going to use these Rust or shortcut modules It's cool to see them go fast wow 3-4 million requests per second on baseline, cool for a benchmark but what I want to see is real web frameworks performance comparison like aspnet vs spring vs quarkus vs fasthttp vs laravel |
That is a framework, request and routing it's the core of any framework, and need to be fast. How you use it later is opinionated. The rest of classes on top are secondary.
Yes, but with ngx-php, we use plain and easy interpreted php, and we can use almost any framework on top. |
|
I have ready 3 full PHP MVC frameworks with Adapteman, than use Workerman, to add to this bench. But some devs, don't want to use these full beasts, they prefer use simpler solutions. |
|
I said here what is production and tuned actually for me. But the language, platform, framework, ... it's a decision of the developer, not from us. Perhaps "production" and "tuned" need to be changed for other more appropriate words. |
|
The production criteria is simple: Default config. If a framework needs to be tuned like nginx this isn't a web framework, it is a web server or something else, web frameworks adapt to the environment they run in and if we allow people to start configuring everything then whatever numbers we see on production are a byproduct of a config crafted to the benchmark nginx and any of its modules cannot fit in the same category as a web framework and I won't mark as framework anything that is not battle tested with proper documentation like any vibe coded framework with 3 stars |
|
Like you want. |
|
Mark it as tuned !! Thank you. |
Completelly contrary to my opinion
|
Not as tuned, will be marked with new category Infrastructure |
|
So OpenRestry will be in the same category. But not Rust frameworks. 🤔 |
|
If workerman is allowed in production so is actix. nginx openresty, ngx_lua, h2o_mruby will fall together with nginx, h2o, traefik, caddy, pingora etc as they are extensions to it and configurations can be applied without having to be marked separately as tuned |
|
and pipeline test will be removed in the future, no need to mention that all the time |
2 participants
Description
The guard function in the nginx access point is removed, as the new crud endpoint need all methods.
And nginx return a 400 Bad method for any incorrect method, before arrive to the locations.
Add backlog to nginx listen, as the server have set
somaxconn 65535.Fix Post problem in the baseline, thanks to checking the docker logs that are in this repo.
Still try to fix the upload problem, but locally we don't have logs to check the errors.
Local platform problems
It's very difficult to test and optimize any framework locally.
Validate.shdon't use the docker build cache, so each time need to download and build the image. It's very slow for each validate, and also we can top the max reqs to docker and github.With
run.shthe build is very fast, almost instantly.benchmark-lite.shstill don't work, fail to build the load generators images.Validate.shshowing the logs when finish, it'll be enough.benchmark-lite.shwork, how we can test the json comp score formula ?It's a very good reference for any framework to balance size and performance.
PR Commands — comment on this PR to trigger (requires collaborator approval):
/benchmark -f <framework>/benchmark -f <framework> -t <test>/benchmark -f <framework> --saveAlways specify
-f <framework>. Results are automatically compared against the current leaderboard.Run benchmarks locally
You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.
Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.