Why does my cache stay invalid after an update?

  • Wildfly 39.0.1
  • Hibernate 6.6.40
  • Infinispan 16.0.5

I’m trying to use the hibernte L2C in a clustered wildfly environment. I have two nodes, behind an apache load balancer, using the standalone-ha configuration that wildfly provides for clustered servers. I’ve stripped my application down to as small as I can, just one Person entity. It’s marked as @Cacheable and my queries all have query.setHint("org.hibernate.cacheable", true)set.
My persistence.xml has cache mode set to ENABLE_SELECTIVE, and hibernate.cache.use_second_level_cache and hibernate.cache.use_query_cache are set to true.

The L2C works great - I can see from the server logs that it makes one actual query per node and then uses the cache instead. Until I mutate an entity. Then, both nodes are invalidated and another query happens when make a reading API call. But it keeps happening - reads don’t use the cache again for what appears to be 60 seconds.

I assume this is unexpected? It’s as if the cache invalidation keeps it invalid, rather than marking it as “needing update”. A subsequent SELECT doesn’t update the cache and mark it as “valid”.

In the log file, it looks like:

2026-08-29 14:07:48,089 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:07:48,089 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:07:48,089 DEBUG [org.hibernate.orm.cache] (default task-2) Returning cached query results
2026-08-29 14:07:49,570 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:07:49,571 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:07:49,571 DEBUG [org.hibernate.orm.cache] (default task-2) Returning cached query results
2026-08-29 14:07:50,335 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:07:50,335 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:07:50,335 DEBUG [org.hibernate.orm.cache] (default task-2) Returning cached query results
# UPDATE ON NODE 2
2026-08-29 14:07:57,364 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:07:57,364 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:07:57,364 DEBUG [org.hibernate.orm.cache] (default task-2) Cached query results were not up-to-date
2026-08-29 14:07:57,365 DEBUG [org.hibernate.orm.cache] (default task-2) Caching query results in region: default-query-results-region; timestamp=1788026877363
2026-08-29 14:08:05,462 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:08:05,462 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:08:05,462 DEBUG [org.hibernate.orm.cache] (default task-2) Cached query results were not up-to-date
2026-08-29 14:08:05,464 DEBUG [org.hibernate.orm.cache] (default task-2) Caching query results in region: default-query-results-region; timestamp=1788026885462
# [...] LOTS OF THESE
2026-08-29 14:08:53,194 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-29 14:08:53,194 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-29 14:08:53,195 DEBUG [org.hibernate.orm.cache] (default task-2) Returning cached query results

There is a timestamp cache that records when a “space” (usually a table) was last updated. The query result cache requires that the cached results were written at T1 where every last write to a table that the query touches happened before T1.
If you have a query like from MyEntity1 e where not exists (select 1 from MyEntity2 e2 where e2.id = e.id), the query has spaces MyEntity1 and MyEntity2. If any non-select operation happens on any of the MyEntity1 and MyEntity2 spaces, the timestamp is updated and hence, the query cache can not be used.
It’s a very coarse grained invalidation mechanism, but without trying to understand what the query does and analyzing old and new values for insert/update/delete statements, it’s next to impossible to do a more fine grained invalidation.

Maybe your application modifies entries in these spaces in the meantime and it just seems like an unnecessary cache miss to you because the data in the query cache is not actually affected by the changes?

Thanks for taking a look at this for me!

I’m pretty sure that I there are no other writes occurring. I am reproducing this with very simple code - just one entity, plus a rest resource and JEE Application class to give me a way to test it. (I’ll paste my code after this)

Is it possible that the invalidating, “dirty” timestamp is set to “now() + delta”? That’s what I seem to see (and delta appears to be 60 seconds).

I notice in the logs that there are entries that look like this:

2026-08-31 10:25:15,677 DEBUG [org.hibernate.cache.internal.TimestampsCacheEnabledImpl] (default task-2) Pre-invalidating space [people], timestamp: 1788186375677
2026-08-31 10:25:15,679 DEBUG [org.hibernate.cache.internal.TimestampsCacheEnabledImpl] (default task-2) Invalidating space [people], timestamp: 1788186315679

And future logs show that hibernate compares against the “pre-invalidating space” timestamp:

# Call UPDATE api
2026-08-31 10:54:16,799 DEBUG [org.hibernate.SQL] (default task-2) update people set age=?,name=? where id=?
2026-08-31 10:54:16,801 DEBUG [org.hibernate.cache.internal.TimestampsCacheEnabledImpl] (default task-2) Pre-invalidating space [people], timestamp: 1788188116801
2026-08-31 10:54:16,811 DEBUG [org.hibernate.cache.internal.TimestampsCacheEnabledImpl] (default task-2) Invalidating space [people], timestamp: 1788188056811
# Call QUERY api
2026-08-31 10:55:13,107 DEBUG [org.hibernate.orm.sql.exec] (default task-2) Reading Query result cache data per CacheMode#isGetEnabled [NORMAL]
2026-08-31 10:55:13,107 DEBUG [org.hibernate.orm.cache] (default task-2) Checking cached query results in region: default-query-results-region
2026-08-31 10:55:13,107 DEBUG [org.hibernate.cache.internal.TimestampsCacheEnabledImpl] (default task-2) [people] last update timestamp: 1788188116801, result set timestamp: 1788188029639
2026-08-31 10:55:13,107 DEBUG [org.hibernate.orm.cache] (default task-2) Cached query results were not up-to-date
2026-08-31 10:55:13,107 DEBUG [org.hibernate.SQL] (default task-2) select p1_0.id,p1_0.age,p1_0.name from people p1_0 order by p1_0.id
2026-08-31 10:55:13,108 DEBUG [org.hibernate.orm.cache] (default task-2) Caching query results in region: default-query-results-region; timestamp=1788188113107

The pre-invalidating timestamp appears to be set to 60 seconds in the future:

jshell> (java.time.LocalDateTime.parse("2026-08-31 10:54:16,801", java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss,SSS")).atZone(java.time.ZoneId.of("America/New_York")).toInstant().toEpochMilli() - 1788188116801L)
$7 ==> -60000

btw - my test runs both instances on the same machine, so I don’t think this can be clock skew. And an exact skew of 60 seconds is hard to imagine :-).


My test code

Entity

package com.example.people;

import jakarta.persistence.Cacheable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;

@Entity
@Cacheable
@Table(name = "people")
public class Person {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(nullable = false)
    private String name;

    @Column(nullable = false)
    private int age;

    public Person() {
    }

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
}

Application:

package com.example.people;

import jakarta.ws.rs.ApplicationPath;

@ApplicationPath("/api")
public class PersonApplication extends Application {
}

Rest Resource:

package com.example.people;

import java.util.List;

import jakarta.enterprise.context.RequestScoped;
import jakarta.persistence.EntityManager;
import jakarta.persistence.PersistenceContext;
import jakarta.persistence.TypedQuery;
import jakarta.transaction.Transactional;
import jakarta.ws.rs.Consumes;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;

@Path("/people")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
@RequestScoped
public class PersonResource {

    @PersistenceContext(unitName = "peopleUnit")
    private EntityManager em;

    @GET
    public List<Person> getAll() {
        TypedQuery<Person> query = em.createQuery("SELECT p FROM Person p ORDER BY p.id", Person.class);
        query.setHint("org.hibernate.cacheable", true);
        return query.getResultList();
    }

    @GET
    @Path("/{id}")
    public Response getById(@PathParam("id") Long id) {
        Person person = em.find(Person.class, id);
        return Response.ok(person).build();
    }

    @GET
    @Path("/name/{name}")
    public List<Person> getByName(@PathParam("name") String name) {
        TypedQuery<Person> query = em.createQuery(
                "SELECT p FROM Person p WHERE p.name = :name ORDER BY p.id", Person.class);
        query.setParameter("name", name);
        query.setHint("org.hibernate.cacheable", true);
        return query.getResultList();
    }

    @POST
    @Transactional
    public Response add(Person person) {
        person.setId(null);
        em.persist(person);

        return Response.status(Response.Status.CREATED).entity(person).build();
    }

    @POST
    @Path("/{id}/age")
    @Consumes(MediaType.TEXT_PLAIN)
    @Transactional
    public Response setAge(@PathParam("id") Long id, String ageText) {
        int
            age = Integer.parseInt(ageText.trim());

        Person person = em.find(Person.class, id);
        person.setAge(age);

        return Response.ok(person).build();
    }

}

Persistence:

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="https://jakarta.ee/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="https://jakarta.ee/xml/ns/persistence
             https://jakarta.ee/xml/ns/persistence/persistence_3_0.xsd"
             version="3.0">

    <persistence-unit name="peopleUnit" transaction-type="JTA">
        <jta-data-source>java:/PeopleDS</jta-data-source>

        <class>com.example.people.Person</class>

        <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
        <properties>
            <property name="jakarta.persistence.schema-generation.database.action" value="create"/>
            
            <property name="hibernate.cache.use_second_level_cache" value="true"/>
            <property name="hibernate.cache.use_query_cache" value="true"/>
            <property name="hibernate.cache.default_cache_concurrency_strategy" value="READ_WRITE"/>
            
        </properties>
    </persistence-unit>
</persistence>

Pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>jakartaee-people</artifactId>
    <version>1.0.0</version>
    <packaging>war</packaging>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <jakartaee.version>10.0.0</jakartaee.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>jakarta.platform</groupId>
            <artifactId>jakarta.jakartaee-api</artifactId>
            <version>${jakartaee.version}</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <finalName>people</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>3.4.0</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

I believe this is a bug and I think I’ve found the issue.

Both GraphBasedActionQueue and ActionQueueLegacy have an invalidateSpaces
method that actually calls preInvalidate :

(The GraphBasedActionQueue invocation is easy to find - this site wont let me put in more than two links …)

It looks like caching queries (at least in the “legacy” ActionQueue implementation - maybe
that’s still the one being used?) will result in that method being called.

I don’t see a place where TimestampCache::invalidate is called afterwards. So it
looks to me like enabling cache querying will result in caches being
effectively disabled for 60 seconds (well, RegionFactory::getTimeout) after any
ActionQueue::executeActions call.

Interesting find. This could be a bug indeed. Please try to create a reproducer with our test case template and if you are able to reproduce the issue, create a bug ticket in our issue tracker and attach that reproducer.

I don’t think I have the hibernate knowledge to get it all down to one unit test. I can reproduce it without the full wildfly environment, but it needs infinispan, and wouldn’t know how to get something like this running without an infinispan.xml and a persistence.cfg.xml. And, I suppose, a separate class for entities. So I can reproduce it with two .java files and 2 config files (and a pom, I suppose).

Can I file a ticket like that?

Sure, something is better than nothing :slight_smile: