Proxy performance confirmation

[makssent]

Hi everyone! 👋

We recently started testing the performance of ShardingSphere-Proxy. We used a profiler and analyzed the results with a flamegraph, but haven’t seen any performance improvements — on the contrary, there’s some regression compared to JDBC. I don’t want to dive into testing advice for now — like “don’t use full routing,” etc. I’m more interested in something else:

👉Could you please share any articles or sources that demonstrate performance improvements with Proxy? Maybe there are materials with visual graphs or metrics showing that Proxy actually performs better (or at least on par) compared to direct database access? Or perhaps you can tell me — does Proxy currently provide any performance benefits in specific scenarios? I’ve often seen mentions that Proxy is slower than JDBC, but I haven’t found any articles that clearly show it outperforming direct database access in real benchmarks.

Thanks! 🙏

[terrymanu]

ShardingSphere-Proxy does indeed perform worse than ShardingSphere-JDBC, and ShardingSphere-JDBC will also be slower than a direct database connected (since it adds some new features). Based on past experience, ShardingSphere-Proxy can be over 30% slower than ShardingSphere-JDBC.
We can explore ways to improve performance. If necessary, you may provide flame graphs to analyze where the performance bottlenecks are together.

[makssent]

ShardingSphere-Proxy does indeed perform worse than ShardingSphere-JDBC, and ShardingSphere-JDBC will also be slower than a direct database connected (since it adds some new features). Based on past experience, ShardingSphere-Proxy can be over 30% slower than ShardingSphere-JDBC. We can explore ways to improve performance. If necessary, you may provide flame graphs to analyze where the performance bottlenecks are together.

I understand and accept that.
It’s logical that with a single shard, there’s an additional routing layer introduced by ShardingSphere, and in that case, Proxy will naturally perform slower than a direct database connection.

However, as I understand it, this overhead should be compensated when the number of shards increases — for example, with 2, 4, 6, or 8 shards, where the load is distributed and the overall effect should balance out the routing costs.
But according to our tests, we don’t see that happening: performance does increase, but not proportionally to the loss introduced by Proxy.
With JDBC, on the other hand, everything works as expected — performance scales up and the improvement is noticeable.

That’s why I wanted to ask — are there any articles, posts, or public results where someone has tested ShardingSphere-Proxy and achieved performance gains?
Perhaps you or the community have conducted such experiments — I just haven’t come across any examples showing Proxy providing an actual performance improvement, only comparisons indicating that Proxy is slower than JDBC.

Right now, we’re deciding whether to continue investing in improving Proxy or to focus on adapting the JDBC variant for our needs.
Of course, we understand that performance depends on many factors — database size, number of shards, hardware configuration, and so on — but it would really help to see a real-world example where Proxy shows a performance improvement as the number of shards increases, compared to a regular database connection without ShardingSphere. That way, we could reproduce such an experiment, verify the results, and make further decisions based on that.

[makssent]

I understand that I could provide you with a flamegraph or other performance metrics, but I think it’s not very useful at this stage — we could keep optimizing endlessly without a clear goal.
What I’d really like to see right now is whether there’s any existing example where Proxy showed a performance improvement on PostgreSQL or MySQL.
If such a case exists, it would confirm that achieving better performance is possible, and we could try to reproduce that experiment or at least use it as a reference point to guide further optimization.

At this point, I’m mostly trying to find a real example where someone actually achieved a performance improvement with Proxy.

P.S. The choice of database shouldn’t matter much here — so far, we haven’t seen any proven performance improvements with Proxy on any database.

[terrymanu]

These are the test results from my current company (SphereEx).
You can consider SphereEx-DBPlusEngine in the diagram as equivalent to ShardingSphere.
When the data volume is sufficiently large and the number of shards is adequate, both JDBC and Proxy show significant performance improvements.
However, the improvement with JDBC is significantly more pronounced compared to Proxy.

[terrymanu]

Here is also a research paper (Apache ShardingSphere: A Holistic and Pluggable Platform for Data Sharding) that includes the results of performance testing.

[makssent]

Thanks for sharing your benchmarks and examples. Did you use the standard MySQL Proxy from ShardingSphere without any custom modifications?

[terrymanu]

Yes.