Hacker News new | past | comments | ask | show | jobs | submit login
Ultra-low-latency, batching and concurrent queue for IPC in Java (github.com/coralblocks)
53 points by eatonphil 9 hours ago | hide | past | favorite | 20 comments





https://github.com/coralblocks/CoralRing/blob/1168b047e0183c...

Am I missing something here or does the BlockingRingConsumer not actually block? And worse doesn't it just return garbage if poll is called without first checking availableToPoll?

The example sure looks like it... https://github.com/coralblocks/CoralRing/blob/main/src/main/...

Which if so isn't this like 1/4th a library for doing IPC? It doesn't seem to do much itself


Weird to see this here! I've used CoralBlocks in the low-latency trading domain previously. Highly recommend. The API is kind, they're very responsive, and the latency is exceptional (and comes with all the basics like thread pinning built-in for convenience)

How does it compare to LMAX Disruptor if you have any experience with both?

They're both similar in design, the main difference is Coral Queue can be used for IPC between JVMs, using a mmap'd file.

See https://github.com/real-logic/aeron (also from the creator of the disruptor)

> CoralBlocks in the low-latency trading domain previously.

Yeah, modern JVM is a true miracle and you can be x5 productive (and safe!) compared to C/C++

Do you have any recommendations for a low latency work queue (with in a jvm)?

I want to spawn millions of micro-second-tasks per second, to worker cores..

I am on a massive cache CPU so memory latency hasnt raised its ugly head yet

EDIT: not LMAX please...


Given the documentation says that this is supposedly to be between JVMs, how do they handle the serialize/deserialize?

They punt on the actual serialization format: https://www.coralblocks.com/index.php/inter-process-communic...

In most applications like this you'll see direct byte manipulation to byte buffers because you want to pull as much performance as possible.

There are fast serialization formats like SBE that people leverage for this as well.


Does anyone have suggestions for something like this, but in Golang?

This is fascinating. I have no idea what something like this would be used for though... what are the use cases?

Basically if you want to schedule workers on a separate JVM, but don't want to pay the latency cost of something like a DB-backed queue or a library with some FFI component.

Our kafka isn’t reliable enough. I need to write data on disk before flushing it to kafka. Can I use this lis to write data to disk and then consume inside same jvm. I need data to live through restarts

Write your data to a MognoDB database. Then use the Kafka Mongo connector to pull data from MongoDB to Kafka.

You could but its not purpose built for that. You'd probably be happier using some other memory mapped file format for that.

Could you please suggest java library for this if you know one?

crazy idea, but SQLite


Can you explain what your issue is with Kafka? What makes it not reliable enough?

Kafka is good. The problem is we don’t have a dedicated person to manage it so sometimes we have kafka outages

There’s a whole conference in Vegas next week that want to sell you something.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: