• Stars
    star
    130
  • Rank 277,575 (Top 6 %)
  • Language
    Go
  • License
    MIT License
  • Created about 6 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

gocraft/work v2 prototype

gocraft/work v2

GoDoc Go Report Card FOSSA Status CircleCI

Please see cmd/ for enqueuer and worker demo.

Improvements

  • queue backend abstraction
    • redis is still the default, but the new design allows custom queue implementation.
  • simplify the keyspace design of redis queue backend
    • The new design uses 1 redis hash per job, and 1 redis sorted set for queue.
    • Interesting read
  • modular
    • The core only catches panics, retries on failure, and waits if a queue is empty.
    • All other functionalities are either removed or moved to separate middlewares.
  • support binary payload/args with message pack.
  • replace built-in UI with prometheus metrics (use grafana if you want dashboard).
  • additional optimizations (alloc + bulk queue ops)
    BenchmarkWorkerRunJob/work_v1_1-8         	    3000	    515957 ns/op
    BenchmarkWorkerRunJob/work_v2_1-8         	    5000	    284516 ns/op
    BenchmarkWorkerRunJob/work_v1_10-8        	    1000	   2136546 ns/op
    BenchmarkWorkerRunJob/work_v2_10-8        	    5000	    367997 ns/op
    BenchmarkWorkerRunJob/work_v1_100-8       	     100	  18234023 ns/op
    BenchmarkWorkerRunJob/work_v2_100-8       	    1000	   1759186 ns/op
    BenchmarkWorkerRunJob/work_v1_1000-8      	      10	 162110100 ns/op
    BenchmarkWorkerRunJob/work_v2_1000-8      	     100	  12646080 ns/op
    BenchmarkWorkerRunJob/work_v1_10000-8     	       1	1691287122 ns/op
    BenchmarkWorkerRunJob/work_v2_10000-8     	      10	 144923087 ns/op
    BenchmarkWorkerRunJob/work_v1_100000-8    	       1	17515722574 ns/op
    BenchmarkWorkerRunJob/work_v2_100000-8    	       1	1502468637 ns/op
    PASS
    ok  	github.com/taylorchu/work	87.901s
  • http server
    • delete job
    • create job
    • get job status
    • get queue metrics (kubernetes autoscaler integration with keda metrics api scaler)
    • OpenAPI spec

License

FOSSA Status