I can query my questdb table(fifteen millions rows) by browser in 50ms, however, by python it takes 50 seconds. What may result in the problem? Thank you

Community
1

I can query my questdb table(fifteenmillions rows) by browser in 50ms, however, by python it takes 50 seconds. What may result in the problem? Thank you.

1 Like
2

Hi @zhchy75 ,

What does your query and schema look like?

How large is the result set?

When you run a query in the web console, it will cap the result set to 10,000 rows per default. Therefore, depending on your query, it will not necessarily read the whole data set.

When running in Python via psycopg2 or HTTP, it is likely running the whole query and returning the results to you.

3

My query is just like: “SELECT * from table;”
Here is my python codes:

def query_questdb_restfull(query_clause, resp_format=‘JSON’):
resp_format = resp_format.upper()
if resp_format == ‘CSV’:
url = f"http://127.0.0.1:9000/exp"
resp = requests.get(url, {‘query’: query_clause}).txt
return resp
else:
url = f"http://127.0.0.1:9000/exec"
resp = requests.get(url, {‘query’: query_clause}).json()
return resp

query_clause = f"SELECT * FROM mytable;"
resp = query_questdb_restfull(query_clause, resp_format=‘JSON’)
print(resp)

So how can I speed up my query by python?

4

Add a limit to your query so you don’t return the full dataset, I.e SELECT * FROM table LIMIT 10000