feat: task workers

[AlliBalliBaba]

This PR adds a new type of worker thread 'task workers'

caddyfile:

frankenphp {
    task_worker path/to/worker/file.php 10 # same config as regular workers
}

from a regular or worker PHP thread:

frankenphp_handle_task('my task'); # send a task

inside of the task worker:

# equivalent to frankenphp_handle_request(), but handling tasks instead
while(frankenphp_handle_task(string $task) { 
    echo $task; // output: 'my task'
}

Allows:

• Offloading work to a separate threadpool that is started at Init()
• Just running any script in a loop (even without calling frankenphp_handle_task)
• Interacting with threads from go via extensions (Add modular threads #1795)

#1795 is also the reason why this feature is marked as experimental for now. Currently tasks are only passed as strings, but it would theoretically be possible to pass anything, even closures (while keeping thread safety in mind).

[dunglas]

Couldn't we merge this with #1795?

I'm pretty sure we can optimize #1795 to not parse the request if not provided by adapting what you have done.

I like the function allowing to process a message asynchronously. Something like that was on my todolist.

[AlliBalliBaba]

Yeah we can merge this with the modular threads logic 👍 feel free to adjust the worker as you need it

[henderkes]

This looks great. That will replace my need for the caddy-supervisor module.

[AlliBalliBaba]

There are still quite a few points open:

• passing any zval to dispatchTask
• returning a zval at the end of the dispatches task
• documentation
• using the same logic for 'worker extensions'
• mechanisms to handle overflowing queues (maybe?)

I'm trying right now to pass any zval. After looking at how it's done in the parallel extension, we probably need to deep copy the values, though it doesn't have to be quite that complicated. Having a secure mechanism to pass zvals between threads might also be quite nice for extensions in general.

We can go ahead with the current state as an initial version and work from there or I can do everything as a big chunk in this PR.

[withinboredom]

I'm trying right now to pass any zval.

I'd think that just like in the parallel extension, it might have some of the same issues? Such as trying to pass an exception through? Can we just use the parallel extension to lean on pre-existing code?

[AlliBalliBaba]

Yeah we probably can, C dependencies are just kind of cumbersome to install and hard to test. We could allow passing anything if ext/parallel is installed, but only allow a string otherwise.

Easiest way is to just go with strings for now.

[dunglas]

Cannot we copy/paste the code (with credits)? It would be better to have a core feature depending on a 3rd-party extension.

[AlliBalliBaba]

I guess the only case where it would make sense to use frankenphp_handle_request() for both HTTP requests and messages is if you actually want a single thread to do both simultaneously. IMO a bad separation of concerns, but I'd be fine with it if you have an actual use case.

[AlliBalliBaba]

Just to clarify again where we go from here:

• would you be fine with separate apis for messages and for requests?
• would you be fine with a separate thread type for messages? (and separate registration in the caddyfile, same config though)
• How would we combine the logic? Merge the PRs and then work in a separate PR or do it all here?

[AlliBalliBaba]

Too many conflicts from main, might revive this with better design at some point.

[tehmaestro]

I think this would be absolutely gold. Is my understanding correct that it could replace libraries like spatie/async or amphp/parallel but using threads?

[henderkes]

No, that would be unrelated. It would be more useful for stuff like queue consumers/workers.

But I agree, it would be very nice to have. Currently I still depend on caddy-supervisor to run these things, it would be neat to have this integrated more cleanly.

[dunglas]

I'm for supporting this too.

This would allow running natively long-running symfony commands such as messenger:consume

[AlliBalliBaba]

Yes this would also enable parallelism by adding a way to wait for task completion.

Would also be super nice to restart workers with messenger:consume or similar automatically on file change. Not sure yet how to best force a script shutdown, maybe somehow with signals?