Is it possible to have dynamic table mapping/schema?
Azure Data Explorer question. Is it possible to have dynamic table mapping/schema? I want to create a table that functions like a Splunk index where it ingests all sorts of data that I can't create a mapping/schema for due to not knowing what the fields are. In Sentinel, custom logs have dynamic schemas.
Answers
-
:wave: , Azure Data Explorer is currently not supported as a destination yet but it is on the roadmap. About your question, I don't think that Azure data explorer provides that functionality. There cannot exist a table that accepts any kind of schema. Any data mismatch might result in empty or incorrect data ingestion https://learn.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings|https://learn.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings
0 -
If it's similar to Log Analytics there is a max limit of 500 columns in a table (I learnt that the hard way when trying to ingest GCP audit logs into Log Analytics)
0 -
We're sending the data through an event hub and created an ingestion point between the event hub and data explorer. Data Explorer is similar to Log Analytics, but the main issue is the schema mapping.
0 -
Try a schemaless destination that supports S3?
0 -
The data explorer tables require a schema from what I could tell. Can you make a schema less table?
0
