01. What is DM&E

90% of the time a ML engineer spends is in actually getting, curating the data and deploying/creating the infrastructure for their models, very little is actually spent writing code

There is a pyramid:

• Wisdom - Applying knowledge insight-fully (using the meteorological knowledge to advise the public about weather hazards, farmers, etc.)
• Knowledge - Information grouped on the meaning(same topic) / Actionable information (meteorological analysis that explains current weather pattern/predicts them)
• Information - Data grouped based on logical patterns/context (Weather report for a region)
• Data - unstructured, objective facts (temperature reading)

Big data has to have the following properties:

• Veracity -> Inconsistencies and uncertainty in data (about handling imperfections -> making in truthful)
• Value -> Extract useful data
• Volume -> Huge amount of data
• Variety -> Different formats of data from various sources
• Velocity -> High speed of accumulation of data