JobTrackerResponsibilities Resource Management (RM) Scheduling: like ten tasks per machine, who is in charge of reducing

ProcessInput data(key-value pairs) -> split -> Map -> shuffle -> Intermediate data(key-value pairs) ->

SparkIt is for full-DAG query processing. Its first-class citizen: RDD. 8 nodes, 16 cores per node and 128 GB of memory

MeasurementsPrefixes: mili: $m$, 0.001,3, micro: $\mu$, 0.000 001, 6, nano: $n$, 9, pico: $p$, 12, femto: $f$, 15,

Use Cases Billions of TB files: the files are relatively small, but the amount is large Object Storage (technology) Ke

HBase: design to run on a scalable cluster of commodity hardware, built on HDFS. Founding paper: Google’s BigTable. Desi

Old Local File SystemFile = Content + Metadata fixed metadata -> fixed Schema organized in a hierarchy files are sto

Type Systemedge vs node labeling: labels are on the edges(JSON)/nodes (XML) Shared properties distinction between atomic

Database prehistory: speaking/singing(expressing information). writing(recording). accounting (processing infor

Table equivalent Conceptstable,collectionattribute,column,field,propertyrow,business object,item,entity,document,recordp