How to await for rows ingested to WAL table to become visible in QuestDB?

Community
,
1

I’m running a single-time QuestDB ingestion for a large data volume. I’m using InfluxDB Line Protocol, and the table uses Write-Ahead Log (WAL). Since inserts are asynchronous, I want to ensure that all ingested rows are visible for read-only queries once I’m done with the ingestion.

Are there any approaches other than running a SELECT count() query and checking the number of visible rows?

2

Running SELECT count(*) FROM my_table is certainly a viable approach. Another approach would be to run the following query over the wal_tables table function:

SELECT 1
FROM wal_tables()
WHERE name = 'my_table' and writerTxn = sequencerTxn;

Here, writerTxn is the last committed transaction available for read-only queries and sequencerTxn is the last transaction committed to WAL. So, if these numbers match for a table, the above query will return a row which means that all rows became visible for queries.

1 Like