Scaling may help with that, indeed – especially replication if you see this slowdown mostly under high load, or sharding if you see it only with large indexes.
You might want to investigate the particular queries causing trouble, though. If the bottleneck is loading data from the database, scaling up your Elasticsearch cluster obviously won’t help. Similarly, there might be some problems with the queries themselves that mean they perform poorly and always will (thinking of regexp predicates in particular, but there are other performance traps).
If it’s about a single document which is too large or with a single field that is too large, adding more Elasticsearch nodes will not help. I’d need to see the error to say for sure what would help.
If it’s about a bulk indexing request that failed to complete in time (timed out, or rejected because the server cannot handle a request at the moment), then tuning Hibernate Search could help, but having more Elasticsearch nodes might help as well, indeed (at least, if you use node discovery).