aaand Query API v2 is live - UI work is coming!
published: 2022-12-12
Let's make it simple: this is a huuge step ahead :-)
~1 month ago we published our new data storage supporting custom events. Of course the collected data - no matter what granularity/quality - worth nothing if we can not query that out. By publishing the two new GET endpoints in our v2 Query API solves that problem. (find the API specs here: Keytiles APIs - Query API)
The new Query API
We learned a lot from the Query API v1 and it had several week points while had limited knowledge too.
Now the goal was not less but fix this. Keep everything as simple as possible while also provide extreme granularity and more and more data if user has the interest - still with an acceptable complexity and response size. These endpoints are about: balance!
Compared to what we had in Query API v1.0 this new one
- Supports the custom events
... of course - this is one of the main points. Support everything the new Storage Engine collects. - Simplifies the query range
Let's face it! The 'tMinusMinutes' parameter in v1.0 just brought confusion to be honest... - Tunable interest
Instead of returning every detail Keytiles knows by default we now just return the most important info: how many events has happened on my site in the query range all together? Period. And if you are interested in more, maybe new vs returning visitors, you can request more details! See the interest parameter description! - Poweful filtering options
Actually this part was not that bad even in 1.0 but now we have more possibilities - just scroll through the *Only parameters! You can filter for eventTypes, tileTypes, userAgentTypes (mobile? desktop? etc), content structure so tileGroupPath, visitorType, language (yes, this is also new :-)), aaand: traffic source. like "social", or even on 2nd level, like "facebook" - Break down of the numbers / different criteria
by introducing the groupBy parameter. Do you want to see the numbers / eventType? Simply do groupBy=eventType and you will get it. Are you querying one day of data and you want to see the numbers / every hour? Then go with groupBy=time:1h. Need both at the same time? Go with groupBy=eventType,time:1h and you get it! But there is much more just check the parameter description! - Limits
to keep the response small as possible. The v1 version always returned data for all Tiles which had even 1 action in the query range. The response was was very unbalanced this way in a sense typically a few Tiles received the 90% of the traffic while tons of Tiles still returned who just received the rest 10% all together. These are less relevant often however they were responsible for the 90% of the response size. And we had huuuge responses even 20Mb or so. Now you can control this. You can say "hey I just need the top 80 Tiles" and you go with limit=80 parameter. You should also check the threshold parameter...
But to achieve all of these possibilities we also had to sacrifice something: human readability. No doubts the v1.0 was more friendly with human eyes. But it was also a huge blocker in many sense. And hey! This is an API right? Which is primarily for machines and not humans anyways ;-) So be it!
What's next / left?
To put it short: rework our TileView and our System UI. So they can benefit from the new backend possibilities.
Since the changes are so big and we have so much more options now on backend probably we need to rethink lots of things from the beginning. :-) But until this happens for sure we will add some useful features just the simple way to our UIs so users can benefit from the new Storage Engine possibilities.
Stay tuned for these changes! Very likely they are coming in 2023 January-February!