1. Working with data.frames (recap)

Loading data from CSV

eyJsYW5ndWFnZSI6InIiLCJzYW1wbGUiOiIjIExvYWQgdGhlIHJlYWRyIHBhY2thZ2VcblxuIyBMb2FkIHRoZSBjb3Vyc2UgZGF0YSBmcm9tIHRoZSBjb3V0cnNlIHdlYnNpdGUgdG8gdGhlIG9iamVjdCAnZGF0YVNldCc6XG4jIGh0dHA6Ly93d3cuamVyb2VuY2xhZXMuYmUvc3RhdGlzdGljc19mb3JfbGluZ3Vpc3RpY3MvZGF0YXNldHMvY2xhc3MzX2NsYWVzXzIwMTcuY3N2XG5cbiMgTG9hZCB0aGUgZHBseXIgcGFja2FnZVxuXG4jIFByaW50IGEgJ2dsaW1wc2UnIG9mIHRoZSBkYXRhU2V0Iiwic29sdXRpb24iOiIjIExvYWQgdGhlIHJlYWRyIHBhY2thZ2VcbmxpYnJhcnkocmVhZHIpXG4jIExvYWQgdGhlIGNvdXJzZSBkYXRhIGZyb20gdGhlIGNvdXJzZSB3ZWJzaXRlIHRvIHRoZSBvYmplY3QgJ2RhdGFTZXQnOlxuIyBodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlxuZGF0YVNldCA8LSByZWFkX2NzdihcImh0dHA6Ly93d3cuamVyb2VuY2xhZXMuYmUvc3RhdGlzdGljc19mb3JfbGluZ3Vpc3RpY3MvZGF0YXNldHMvY2xhc3MzX2NsYWVzXzIwMTcuY3N2XCIpXG4jIExvYWQgdGhlIGRwbHlyIHBhY2thZ2VcbmxpYnJhcnkoZHBseXIpXG4jIFByaW50IGEgJ2dsaW1wc2UnIG9mIHRoZSBkYXRhU2V0XG5nbGltcHNlKGRhdGFTZXQpIiwic2N0IjoidGVzdF9vYmplY3QoXCJkYXRhU2V0XCIpXG50ZXN0X2xpYnJhcnlfZnVuY3Rpb24oXCJyZWFkclwiLCBcIk1ha2Ugc3VyZSB0byBjYWxsIHRoZSAncmVhZHInIHBhY2thZ2UhXCIpXG50ZXN0X2xpYnJhcnlfZnVuY3Rpb24oXCJkcGx5clwiLCBcIk1ha2Ugc3VyZSB0byBjYWxsIHRoZSAnZHBseXInIHBhY2thZ2UhXCIpXG50ZXN0X291dHB1dF9jb250YWlucyhcImdsaW1wc2UoZGF0YVNldClcIiwgICBpbmNvcnJlY3RfbXNnID0gXCJNYWtlIHN1cmUgdG8gcHJpbnQgYSAnZ2xpbXBzZScgb2YgdGhlIGRhdGEhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

Subsetting data.frames (1/4)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiLCBsb2NhbGU9bG9jYWxlKGVuY29kaW5nID0gXCJVVEYtOFwiKSkiLCJzYW1wbGUiOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG5cbiMgUHJpbnQgdGhlIGZpcnN0IDI1IHZhbHVlcyBvZiBpdHMgbm91biBjb2x1bW4iLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBQcmludCB0aGUgZmlyc3QgMjUgdmFsdWVzIG9mIGl0cyBub3VuIGNvbHVtblxuaGVhZChkYXRhU2V0JG5vdW4sMjUpIiwic2N0IjoidGVzdF9vdXRwdXRfY29udGFpbnMoXCJoZWFkKGRhdGFTZXQkbm91biwgMjUpXCIsICBpbmNvcnJlY3RfbXNnID0gXCJNYWtlIHN1cmUgdG8gcHJpbnQgdGhlIGZpcnN0IDI1IG5vdW5zXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

Subsetting data.frames (2/4)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBQcmludCB0aGUgZmlyc3QgNiB2YWx1ZXMgb2Ygbm91biB0aGF0IGhhdmUgbm91bl9sZW5ndGggZXF1YWwgdG8gNiIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIFByaW50IHRoZSBmaXJzdCA2IHZhbHVlcyBvZiBub3VuIHRoYXQgaGF2ZSBub3VuX2xlbmd0aCBlcXVhbCB0byA2XG5oZWFkKGRhdGFTZXRbZGF0YVNldCRub3VuX2xlbmd0aD09NixjKFwibm91blwiKV0pIiwic2N0IjoidGVzdF9vdXRwdXRfY29udGFpbnMoJ2hlYWQoZGF0YVNldFtkYXRhU2V0JG5vdW5fbGVuZ3RoPT02LGMoXCJub3VuXCIpXSknLCBpbmNvcnJlY3RfbXNnPSBcIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgZmlyc3Qgc2l4IHJvd3NcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

Subsetting data.frames (3/4)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBQcmludCB0aGUgbGFzdCAyMCB2YWx1ZXMgJ25vdW4nIHRoYXQgaGF2ZSBhIG5vdW5fbGVuZ3RoIHNtYWxsZXIgdGhhbiA1XG5cbiMgQ3JlYXRlIGEgbmV3IGRhdGEuZnJhbWUsIGNhbGxlZCBzaG9ydE5vdW5zLCB3aGljaCBob2xkcyBhbGwgcm93cyBhbmQgYWxsIGNvbHVtbnMgb2YgZGF0YVNldCwgd2hlcmUgbm91bl9sZW5ndGggaXMgc21hbGxlciB0aGFuIG9yIGVxdWFsIHRvIDEwXG5cbiMgUHJpbnQgdGhlIDEwdGgsIHRoZSB0aGlydGlldGgsIGFuZCB0aGUgMTAwdGggcm93IG9mIHRoaXMgZGF0YS5mcmFtZSIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIFByaW50IHRoZSBsYXN0IDIwIHZhbHVlcyAnbm91bicgdGhhdCBoYXZlIGEgbm91bl9sZW5ndGggc21hbGxlciB0aGFuIDVcbnRhaWwoZGF0YVNldFtkYXRhU2V0JG5vdW5fbGVuZ3RoIDwgNSxjKFwibm91blwiKV0sMjApXG5cbiMgQ3JlYXRlIGEgbmV3IGRhdGEuZnJhbWUsIGNhbGxlZCBzaG9ydE5vdW5zLCB3aGljaCBob2xkcyBhbGwgcm93cyBhbmQgYWxsIGNvbHVtbnMgb2YgZGF0YVNldCwgd2hlcmUgbm91bl9sZW5ndGggaXMgc21hbGxlciB0aGFuIG9yIGVxdWFsIHRvIDEwXG5zaG9ydE5vdW5zIDwtIGRhdGFTZXRbZGF0YVNldCRub3VuX2xlbmd0aCA8PSAxMCxdXG5cbiMgUHJpbnQgdGhlIGZpcnN0LCB0aGUgc2Vjb25kLCBhbmQgdGhlIDUwdGggcm93IG9mIHRoaXMgZGF0YS5mcmFtZVxuc2hvcnROb3Vuc1tjKDEwLCAzMCwgMTAwKSxdIiwic2N0IjoiIHRlc3Rfb3V0cHV0X2NvbnRhaW5zKCd0YWlsKGRhdGFTZXRbZGF0YVNldCRub3VuX2xlbmd0aCA8IDUsIGMoXCJub3VuXCIpXSwyMCknLCAgaW5jb3JyZWN0X21zZyAgPSBcIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgbGFzdCAyMCB2YWx1ZXMgb2Ygbm91blwiKVxudGVzdF9vYmplY3QoXCJzaG9ydE5vdW5zXCIpXG50ZXN0X291dHB1dF9jb250YWlucyhcInNob3J0Tm91bnNbYygxMCwgMzAsIDEwMCksXVwiLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIHByaW50IHRoZSBmaXJzdCwgdGhlIHNlY29uZCwgYW5kIHRoZSA1MHRoIHJvdyBvZiBsb3dGcmVxdWVuY3lcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

Subsetting data.frames (4/4)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmxpYnJhcnkoZHBseXIpXG5kYXRhU2V0IDwtIHJlYWRfY3N2KFwiaHR0cDovL3d3dy5qZXJvZW5jbGFlcy5iZS9zdGF0aXN0aWNzX2Zvcl9saW5ndWlzdGljcy9kYXRhc2V0cy9jbGFzczNfY2xhZXNfMjAxNy5jc3ZcIikiLCJzYW1wbGUiOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG4jIFRoZSBwYWNrYWdlIGRwbHlyIGlzIGFscmVhZHkgbG9hZGVkXG5cbiMgRXh0cmFjdCBhbGwgdmFsdWVzIG9mIGBub3VuYCB3aXRoIG5vdW5fbGVuZ3RoIHNtYWxsZXIgdGhhbiA1LiBQcmludCBvbmx5IHRoZSBmaXJzdCAxNSBub3Vuc1xuXG4jIEV4dHJhY3QgdGhlIG5vdW5fbGVuZ3RoIG9mIHRoZSBmb2xsb3dpbmcgbm91bnM6IFwicmVzdHJpY2Npb25lc1wiIFwiZm90b3NcIiBcIm9saXZhc1wiICBcInBlcmlvZGlzdGFzXCIgXCJjb3NlY2hhZG9yYXNcIiBcblxuIyBBZGQgYSBuZXcgdmFyaWFibGUgdG8gdGhlIGRhdGEuZnJhbWUsIGNhbGxlZCBtZWFuTGVuZ3RoIHdoaWNoIHN0b3JlcyB0aGUgbWVhbiBvZiBub3VuX2xlbmd0aFxuXG4jIFByaW50IGEgc3VtbWFyeSBvZiB0aGUgbW9kaWZpZWQgZGF0YS5GcmFtZSIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuIyBUaGUgcGFja2FnZSBkcGx5ciBpcyBhbHJlYWR5IGxvYWRlZFxuXG4jIEV4dHJhY3QgYWxsIHZhbHVlcyBvZiBgbm91bmAgd2l0aCBub3VuX2xlbmd0aCBzbWFsbGVyIHRoYW4gNS4gUHJpbnQgb25seSB0aGUgZmlyc3QgMTUgd29yZHNcbmhlYWQoZGF0YVNldFtkYXRhU2V0JG5vdW5fbGVuZ3RoIDwgNSxjKFwibm91blwiKV0sIDE1KVxuXG4jIEV4dHJhY3QgdGhlIG5vdW5fbGVuZ3RoIG9mIHRoZSBmb2xsb3dpbmcgbm91bnM6IFwicmVzdHJpY2Npb25lc1wiIFwiZm90b3NcIiBcIm9saXZhc1wiICBcInBlcmlvZGlzdGFzXCIgXCJjb3NlY2hhZG9yYXNcIiBcbmRhdGFTZXRbZGF0YVNldCRub3VuICVpbiUgYyhcInJlc3RyaWNjaW9uZXNcIiwgXCJmb3Rvc1wiLCBcIm9saXZhc1wiLCBcInBlcmlvZGlzdGFzXCIsXCJjb3NlY2hhZG9yYXNcIiksIGMoXCJub3VuX2xlbmd0aFwiKV1cblxuIyBBZGQgYSBuZXcgdmFyaWFibGUgdG8gdGhlIGRhdGEuZnJhbWUsIGNhbGxlZCBGcmVxQnlUd28gd2hpY2ggc3RvcmVzIHRoZSBGcmVxIG9mIHRoZSB3b3JkLCBkaXZpZGVkIGJ5IHR3b1xuZGF0YVNldCRtZWFuTGVuZ3RoIDwtIG1lYW4oZGF0YVNldCRub3VuX2xlbmd0aCwgbmEucm09VFJVRSlcblxuIyBQcmludCBhIGdsaW1wc2Ugb2YgdGhlIG1vZGlmaWVkIGRhdGEuRnJhbWVcbmdsaW1wc2UoZGF0YVNldCkiLCJzY3QiOiJ0ZXN0X291dHB1dF9jb250YWlucygnaGVhZChkYXRhU2V0W2RhdGFTZXQkbm91bl9sZW5ndGggPCA1LGMoXCJub3VuXCIpXSwgMTUpJywgaW5jb3JyZWN0X21zZyA9ICBcIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgZmlyc3QgMTUgdmFsdWVzIG9mICdub3VuJyB3aGVyZSBub3VuX2xlbmd0aCA8IDVcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKCdkYXRhU2V0W2RhdGFTZXQkV29yZCAlaW4lIGMoXCJyZXN0cmljY2lvbmVzXCIsIFwiZm90b3NcIiwgXCJvbGl2YXNcIiwgXCJwZXJpb2Rpc3Rhc1wiLFwiY29zZWNoYWRvcmFzXCIpLCBjKFwibm91bl9sZW5ndGhcIildJywgIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBleHRyYWN0IHRoZSBMZW5ndGggYW5kIHRoZSBGcmVxIG9mIHRoZSB3b3JkcyBpbiB0aGUgbGlzdFwiKVxudGVzdF9vdXRwdXRfY29udGFpbnMoJ2dsaW1wc2UoZGF0YVNldCknLCBpbmNvcnJlY3RfbXNnID0gIFwiTWFrZSBzdXJlIHRvIHByaW50IGEgZ2xpbXBzZSBvZiB0aGUgbW9kaWZpZWQgZGF0YS5mcmFtZSFcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQgam9iIVwiKSJ9

2. The normal distribution (recap)

  • ANALYZE:
  • Are the characters_before_noun and noun_length variables distributed normally?
  • Why do you think so?

2.1 The characters_before_noun variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcbiMgQ2FsY3VsYXRlIGFuZCBjb21wYXJlIHRoZSBtZWFuIG9mIGNoYXJhY3RlcnNfYmVmb3JlX25vdW4gdG8gdGhlIG1lZGlhbiBvZiBjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuXG5cbiMgTG9hZCB0aGUgZ2dwbG90MiBwYWNrYWdlXG5cbiMgRHJhdyBhIHFxcGxvdCB3aXRoIGEgcXFsaW5lIG9mIHRoZSBjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuIHZhcmlhYmxlXG5cbiMgRHJhdyBhIGRlbnNpdHkgcGxvdCBvZiB0aGUgY2hhcmFjdGVyc19iZWZvcmVfbm91biB2YXJpYWJsZVxuXG4jIFBlcmZvcm0gYSBTaGFwaXJvLVdpbGsgdGVzdFxuXG4jIFRISU5LOiBpcyB0aGUgY2hhcmFjdGVyc19iZWZvcmVfbm91biB2YXJpYWJsZSBkaXN0cmlidXRlZCBub3JtYWxseT8gWW91IHdpbGwgZmluZCB0aGUgY29ycmVjdCBhbnN3ZXIgb24gdGhlIFNvbHV0aW9uIHRhYi4gIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG4jIENhbGN1bGF0ZSBhbmQgY29tcGFyZSB0aGUgbWVhbiBvZiBjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuIHRvIHRoZSBtZWRpYW4gb2YgY2hhcmFjdGVyc19iZWZvcmVfbm91blxubWVhbihkYXRhU2V0JGNoYXJhY3RlcnNfYmVmb3JlX25vdW4pXG5tZWRpYW4oZGF0YVNldCRjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuKVxuIyBMb2FkIHRoZSBnZ3Bsb3QyIHBhY2thZ2VcbmxpYnJhcnkoZ2dwbG90MilcbiMgRHJhdyBhIHFxcGxvdCB3aXRoIGEgcXFsaW5lIG9mIHRoZSBjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuIHZhcmlhYmxlXG5xcW5vcm0oZGF0YVNldCRjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuKVxucXFsaW5lKGRhdGFTZXQkY2hhcmFjdGVyc19iZWZvcmVfbm91bilcbiMgRHJhdyBhIGRlbnNpdHkgcGxvdCBvZiB0aGUgY2hhcmFjdGVyc19iZWZvcmVfbm91biB2YXJpYWJsZVxuZ2dwbG90KGRhdGFTZXQsIGFlcyh4PWNoYXJhY3RlcnNfYmVmb3JlX25vdW4pKSArIGdlb21fbGluZShzdGF0PVwiZGVuc2l0eVwiKSArIGdlb21fdmxpbmUoYWVzKHhpbnRlcmNlcHQgPSBtZWFuKGRhdGFTZXQkY2hhcmFjdGVyc19iZWZvcmVfbm91bikpLCBjb2xvcj1cInJlZFwiKVxuIyBQZXJmb3JtIGEgU2hhcGlyby1XaWxrIHRlc3RcbnNoYXBpcm8udGVzdChkYXRhU2V0JGNoYXJhY3RlcnNfYmVmb3JlX25vdW4pXG4jIFRISU5LOiBpcyB0aGUgY2hhcmFjdGVyc19iZWZvcmVfbm91biB2YXJpYWJsZSBkaXN0cmlidXRlZCBub3JtYWxseT8gWW91IHdpbGwgZmluZCB0aGUgY29ycmVjdCBhbnN3ZXIgb24gdGhlIFNvbHV0aW9uIHRhYi4gXG4jIFRoZSBjb3JyZWN0IGFuc3dlciBpcyBubzogXG4jIC0gVGhlIG1lYW4gYW5kIHRoZSBtZWRpYW4gYXJlIG5vdCB2ZXJ5IGNsb3NlYnkgbnVtZXJpY2FsbHlcbiMgLSBUaGUgZG90cyBkbyBub3QgZm9sbG93IHRoZSBsaW5lIGluIHRoZSBxcXBsb3RcbiMgLSBUaGUgbWVhbiBpcyBub3QgaW4gdGhlIG1pZGRsZSBvZiB0aGUgZGlzdHJpYnV0aW9uIGFuZCB0aGUgbGFyZ2VzdCBhbW91bnQgb2YgdmFsdWVzIGRvIG5vdCBvY2N1ciBhcm91bmQgdGhlIG1lYW5cbiMgLSBUaGUgc2hhcGlyby50ZXN0IGhhcyBhIHAgPCAwLjA1Iiwic2N0IjoidGVzdF9saWJyYXJ5X2Z1bmN0aW9uKFwiZ2dwbG90MlwiLCBpbmNvcnJlY3RfbXNnID0gXCJEb24ndCBmb3JnZXQgdG8gbG9hZCB0aGUgZ2dwbG90MiBwYWNrYWdlXCIpXG50ZXN0X291dHB1dF9jb250YWlucyhcIm1lYW4oZGF0YVNldCRjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuKVwiLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGNhbGN1bGF0ZSB0aGUgbWVhblwiKVxudGVzdF9vdXRwdXRfY29udGFpbnMoXCJtZWRpYW4oZGF0YVNldCRjaGFyYWN0ZXJzX2JlZm9yZV9ub3VuKVwiLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGNhbGN1bGF0ZSB0aGUgbWVkaWFuXCIpXG50ZXN0X2Z1bmN0aW9uKFwicXFub3JtXCIsIGFyZ3MgPSBjKFwieVwiKSwgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGRyYXcgYSBxcXBsb3QhXCIpXG50ZXN0X2Z1bmN0aW9uKFwicXFsaW5lXCIsIGFyZ3MgPSBjKFwieVwiKSwgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGFkZCBhIHFxbGluZSB0byB5b3VyIHFxcGxvdCFcIilcbnRlc3RfZ2dwbG90KGluZGV4PTEsIGFsbF9mYWlsX21zZyA9IFwiQ29tcHV0ZXIgc2F5cyBuby4gVGhpcyBwbG90IGlzIGNvbXBsZXRlbHkgd3JvbmdcIiwgZGF0YV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBpbmNsdWRlIHRoZSBkYXRhIGluIHRoZSBnZ3Bsb3QgY2FsbFwiLCBhZXNfZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgZGVmaW5lIHRoZSBjb3JyZWN0IGFlc3RoZXRpY3MgKGFlcylcIiwgZ2VvbV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBkZWZpbmUgdGhlIGNvcnJlY3QgcGxvdHRpbmcgZnVuY3Rpb24gKGdlb21fKVwiKVxuIHRlc3Rfb3V0cHV0X2NvbnRhaW5zKFwic2hhcGlyby50ZXN0KGRhdGFTZXQkY2hhcmFjdGVyc19iZWZvcmVfbm91bilcIiwgIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBwZXJmb3JtIGEgU2hhcGlyby1XaWxrIHRlc3QhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IGpvYiFcIikifQ==

2.2 The noun_length variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcbiMgQ2FsY3VsYXRlIGFuZCBjb21wYXJlIHRoZSBtZWFuIG9mIG5vdW5fbGVuZ3RoIHRvIHRoZSBtZWRpYW4gb2Ygbm91bl9sZW5ndGhcblxuIyBMb2FkIHRoZSBnZ3Bsb3QyIHBhY2thZ2VcblxuIyBEcmF3IGEgcXFwbG90IHdpdGggYSBxcWxpbmUgb2YgdGhlIG5vdW5fbGVuZ3RoIHZhcmlhYmxlXG5cbiMgRHJhdyBhIGRlbnNpdHkgcGxvdCBvZiB0aGUgbm91bl9sZW5ndGggdmFyaWFibGVcblxuIyBQZXJmb3JtIGEgU2hhcGlyby1XaWxrIHRlc3RcblxuIyBUSElOSzogaXMgdGhlIG5vdW5fbGVuZ3RoIHZhcmlhYmxlIGRpc3RyaWJ1dGVkIG5vcm1hbGx5PyBZb3Ugd2lsbCBmaW5kIHRoZSBjb3JyZWN0IGFuc3dlciBvbiB0aGUgU29sdXRpb24gdGFiLiAiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcbiMgQ2FsY3VsYXRlIGFuZCBjb21wYXJlIHRoZSBtZWFuIG9mIG5vdW5fbGVuZ3RoIHRvIHRoZSBtZWRpYW4gb2Ygbm91bl9sZW5ndGhcbm1lYW4oZGF0YVNldCRub3VuX2xlbmd0aClcbm1lZGlhbihkYXRhU2V0JG5vdW5fbGVuZ3RoKVxuIyBMb2FkIHRoZSBnZ3Bsb3QyIHBhY2thZ2VcbmxpYnJhcnkoZ2dwbG90MilcbiMgRHJhdyBhIHFxcGxvdCB3aXRoIGEgcXFsaW5lIG9mIHRoZSBub3VuX2xlbmd0aCB2YXJpYWJsZVxucXFub3JtKGRhdGFTZXQkbm91bl9sZW5ndGgpXG5xcWxpbmUoZGF0YVNldCRub3VuX2xlbmd0aClcbiMgRHJhdyBhIGRlbnNpdHkgcGxvdCBvZiB0aGUgbm91bl9sZW5ndGggdmFyaWFibGVcbmdncGxvdChkYXRhU2V0LCBhZXMoeD1ub3VuX2xlbmd0aCkpICsgZ2VvbV9saW5lKHN0YXQ9XCJkZW5zaXR5XCIpICsgZ2VvbV92bGluZShhZXMoeGludGVyY2VwdCA9IG1lYW4oZGF0YVNldCRub3VuX2xlbmd0aCkpLCBjb2xvcj1cInJlZFwiKVxuIyBQZXJmb3JtIGEgU2hhcGlyby1XaWxrIHRlc3RcbnNoYXBpcm8udGVzdChkYXRhU2V0JG5vdW5fbGVuZ3RoKVxuIyBUSElOSzogaXMgdGhlIG5vdW5fbGVuZ3RoIHZhcmlhYmxlIGRpc3RyaWJ1dGVkIG5vcm1hbGx5PyBZb3Ugd2lsbCBmaW5kIHRoZSBjb3JyZWN0IGFuc3dlciBvbiB0aGUgU29sdXRpb24gdGFiLiBcbiMgVGhlIGNvcnJlY3QgYW5zd2VyIGlzIG5vOiBcbiMgLSBUaGUgZG90cyBhdCB0aGUgYm90dG9tIGFuZCB0aGUgdG9wIG9mIHRoZSBwbG90IGRvIG5vdCBmb2xsb3cgdGhlIGxpbmUgaW4gdGhlIHFxcGxvdFxuIyAtIFRoZSBtZWFuIGlzIG5vdCBpbiB0aGUgbWlkZGxlIG9mIHRoZSBkaXN0cmlidXRpb24gYW5kIHRoZSBkaXN0cmlidXRpb24gaXMgbm90IHN5bW1ldHJpY2FsXG4jIC0gVGhlIHNoYXBpcm8udGVzdCBoYXMgYSBwIDwgMC4wNSwgc28gd2UgY2FuIGFzc3VtZSB0aGF0IHRoZSBkYXRhIGFyZSBub3Qgbm9ybWFsbHkgZGlzdHJpYnV0ZWQiLCJzY3QiOiJ0ZXN0X2xpYnJhcnlfZnVuY3Rpb24oXCJnZ3Bsb3QyXCIsIGluY29ycmVjdF9tc2cgPSBcIkRvbid0IGZvcmdldCB0byBsb2FkIHRoZSBnZ3Bsb3QyIHBhY2thZ2VcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKFwibWVhbihkYXRhU2V0JG5vdW5fbGVuZ3RoKVwiLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGNhbGN1bGF0ZSB0aGUgbWVhblwiKVxudGVzdF9vdXRwdXRfY29udGFpbnMoXCJtZWRpYW4oZGF0YVNldCRub3VuX2xlbmd0aClcIiwgIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBjYWxjdWxhdGUgdGhlIG1lZGlhblwiKVxudGVzdF9mdW5jdGlvbihcInFxbm9ybVwiLCBhcmdzID0gYyhcInlcIiksIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBkcmF3IGEgcXFwbG90IVwiKVxudGVzdF9mdW5jdGlvbihcInFxbGluZVwiLCBhcmdzID0gYyhcInlcIiksIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBhZGQgYSBxcWxpbmUgdG8geW91ciBxcXBsb3QhXCIpXG50ZXN0X2dncGxvdChpbmRleD0xLCBhbGxfZmFpbF9tc2cgPSBcIkNvbXB1dGVyIHNheXMgbm8uIFRoaXMgcGxvdCBpcyBjb21wbGV0ZWx5IHdyb25nXCIsIGRhdGFfZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgaW5jbHVkZSB0aGUgZGF0YSBpbiB0aGUgZ2dwbG90IGNhbGxcIiwgYWVzX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGRlZmluZSB0aGUgY29ycmVjdCBhZXN0aGV0aWNzIChhZXMpXCIsIGdlb21fZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgZGVmaW5lIHRoZSBjb3JyZWN0IHBsb3R0aW5nIGZ1bmN0aW9uIChnZW9tXylcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKFwic2hhcGlyby50ZXN0KGRhdGFTZXQkbm91bl9sZW5ndGgpXCIsICBpbmNvcnJlY3RfbXNnID0gXCJNYWtlIHN1cmUgdG8gcGVyZm9ybSBhIFNoYXBpcm8tV2lsayB0ZXN0IVwiKVxuc3VjY2Vzc19tc2coXCJHcmVhdCBqb2IhXCIpIn0=

3. Outliers

  • The previous excercises have shown that:
  • noun_length is mostly normally distributed, except for a few outliers in the right tail of the distribution, which draw the mean to the right. If we exclude these values, it will be distributed normally
  • number_of_characters_before_noun has a power-law distribution. We will need a logarithmic transformation to correct its distribution

3.1 Removing outliers from noun_length

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBBZGQgYSBuZXcgY29sdW1uIHRvIHRoZSBkYXRhU2V0IGNhbGxlZCBgbm91bl9sZW5ndGhfenNjb3JlYCwgaW4gd2hpY2ggeW91IHN0b3JlIHotc2NvcmVzIGZvciBub3VuX2xlbmd0aFxuXG4jIE92ZXJ3cml0ZSBgbm91bl9sZW5ndGhfenNjb3JlYCBzbyB0aGF0IGl0IGNvbnRhaW5zIHRoZSBhYnNvbHV0ZSB2YWx1ZXMgb2YgdGhlIHotc2NvcmVzXG5cbiMgUmVtb3ZlIGFsbCBvdXRsaWVycyBmcm9tIGRhdGFTZXQuIFJlbWVtYmVyIHRoYXQgb3V0bGllcnMgYXJlIHZhbHVlcyB3aXRoIGFic29sdXRlIHotc2NvcmVzIG9mIHR3byBvciBtb3JlXG5cbiMgUHJpbnQgYSBzdW1tYXJ5IG9mIHRoZSBub3VuX2xlbmd0aCBjb2x1bW5cblxuIyBEcmF3IGEgcXFwbG90IG9mIG5vdW5fbGVuZ3RoIGFuZCBhZGQgYSBxcWxpbmUgXG5cbiMgQU5BTFlaRTogRG9lcyB0aGlzIGZpeCB0aGUgcHJvYmxlbT8iLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBBZGQgYSBuZXcgY29sdW1uIHRvIHRoZSBkYXRhU2V0IGNhbGxlZCBgbm91bl9sZW5ndGhfenNjb3JlYCwgaW4gd2hpY2ggeW91IHN0b3JlIHotc2NvcmVzIGZvciBub3VuX2xlbmd0aFxuZGF0YVNldCRub3VuX2xlbmd0aF96c2NvcmUgPC0gc2NhbGUoZGF0YVNldCRub3VuX2xlbmd0aClcblxuIyBPdmVyd3JpdGUgYG5vdW5fbGVuZ3RoX3pzY29yZWAgc28gdGhhdCBpdCBjb250YWlucyB0aGUgYWJzb2x1dGUgdmFsdWVzIG9mIHRoZSB6LXNjb3Jlc1xuZGF0YVNldCRub3VuX2xlbmd0aF96c2NvcmUgPC0gYWJzKGRhdGFTZXQkbm91bl9sZW5ndGhfenNjb3JlKVxuIyBSZW1vdmUgYWxsIG91dGxpZXJzIGZyb20gZGF0YVNldC4gUmVtZW1iZXIgdGhhdCBvdXRsaWVycyBhcmUgdmFsdWVzIHdpdGggYWJzb2x1dGUgei1zY29yZXMgb2YgdHdvIG9yIG1vcmVcbmRhdGFTZXQgPC0gZGF0YVNldFtkYXRhU2V0JG5vdW5fbGVuZ3RoX3pzY29yZSA8PSAyLCBdXG5cbiMgUHJpbnQgYSBzdW1tYXJ5IG9mIHRoZSBub3VuX2xlbmd0aCBjb2x1bW5cbnN1bW1hcnkoZGF0YVNldCRub3VuX2xlbmd0aClcbiMgRHJhdyBhIHFxcGxvdCBvZiBub3VuX2xlbmd0aCBhbmQgYWRkIGEgcXFsaW5lIFxucXFub3JtKGRhdGFTZXQkbm91bl9sZW5ndGgpXG5xcWxpbmUoZGF0YVNldCRub3VuX2xlbmd0aClcblxuIyBBTkFMWVpFOiBEb2VzIHRoaXMgZml4IHRoZSBwcm9ibGVtP1xuIyBUaGUgZGlzdHJpYnV0aW9uIGlzIG11Y2ggY2xvc2VyIHRvIGEgbm9ybWFsIGRpc3RyaWJ1dGlvbiwgYnV0IGl0IGlzIHN0aWxsIGZhciBmcm9tIGlkZWFsIiwic2N0IjoidGVzdF9vdXRwdXRfY29udGFpbnMoXCJzdW1tYXJ5KGRhdGFTZXQkbm91bl9sZW5ndGgpXCIsIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBwcmludCBhIHN1bW1hcnkgb2YgdGhlIGZpbHRlcmVkIG5vdW5fbGVuZ3RoIGNvbHVtblwiKVxudGVzdF9mdW5jdGlvbihcInFxbm9ybVwiLCBhcmdzID0gYyhcInlcIiksIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBkcmF3IGEgcXFwbG90IVwiKVxudGVzdF9mdW5jdGlvbihcInFxbGluZVwiLCBhcmdzID0gYyhcInlcIiksIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBhZGQgYSBxcWxpbmUgdG8geW91ciBxcXBsb3QhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IGpvYiFcIikifQ==

4. Factors

4.1 Factor levels

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gc3RhdGUgdG8gYSBmYWN0b3JcblxuIyBQcmludCBpdHMgbGV2ZWxzIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG5cbiMgQ29udmVydCB0aGUgY29sdW1uIHN0YXRlIHRvIGEgZmFjdG9yXG5kYXRhU2V0JHN0YXRlIDwtIGFzLmZhY3RvcihkYXRhU2V0JHN0YXRlKVxuIyBQcmludCBpdHMgbGV2ZWxzXG5sZXZlbHMoZGF0YVNldCRzdGF0ZSkiLCJzY3QiOiIgdGVzdF9vdXRwdXRfY29udGFpbnMoJ2xldmVscyhkYXRhU2V0JHN0YXRlKScsICBpbmNvcnJlY3RfbXNnICA9IFwiTWFrZSBzdXJlIHRvIHByaW50IHRoZSBsZXZlbHMgb2YgdGhlIGZhY3RvciFcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

4.2 Recoding factors

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSAiLCJzYW1wbGUiOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG4jIExvYWQgdGhlIGRwbHlyIHBhY2thZ2VcblxuIyBDb252ZXJ0IGFsbCBjaGFyYWN0ZXIgY29sdW1ucyB0byBmYWN0b3JcblxuIyBUaGUgYHN0YXRlYCBjb2x1bW4gaGFzIHF1aXRlIHNvbWUgZHVwbGljYXRlczogXG4jIFwiQW5kYWx1Y1x1MDBlZGFcIiBhbmQgXCJBbmRhbHVzaWFcIiwgXCJBcmFnb25cIiBhbmQgXCJBcmFnXHUwMGYzblwiLCBcIkNhc3RpbGUtTGEgTWFuY2hhXCIgYW5kIFwiQ2FzdGlsbGEtTGEtTWFuY2hhXCIsIFwiQ2FzdGlsZSBhbmQgTGVcdTAwZjNuXCIgYW5kIFwiQ2FzdGlsbGEgeSBMZVx1MDBmM25cIiwgXCJDYXRhbG9uaWFcIiBhbmQgXCJDYXRhbHVcdTAwZjFhXCIsIFwiQ29tbXVuaXR5IG9mIE1hZHJpZFwiIGFuZCBcIkNvbXVuaWRhZCBkZSBNYWRyaWRcIiwgXCJWYWxlbmNpYW4gQ29tbXVuaXR5XCIgYW5kIFwiQ29tdW5pZGFkIFZhbGVuY2lhbmFcIiwgXCJCYXNxdWUgQ291bnRyeVwiIGFuZCBcIlBhXHUwMGVkcyBWYXNjb1wiXG4jIFJlY29kZSBhbGwgU3BhbmlzaCBuYW1lcyB0byB0aGVpciBFbmdsaXNoIGFsdGVybmF0aXZlXG5cbiMgUHJpbnQgdGhlIG5ldyBsZXZlbHMgb2YgdGhlIGBzdGF0ZWAgY29sdW1uIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG4jIExvYWQgdGhlIGRwbHlyIHBhY2thZ2VcbmxpYnJhcnkoZHBseXIpXG4jIENvbnZlcnQgYWxsIGNoYXJhY3RlciBjb2x1bW5zIHRvIGZhY3RvclxuZGF0YVNldCA8LSBtdXRhdGVfaWYoZGF0YVNldCwgaXMuY2hhcmFjdGVyLCBhcy5mYWN0b3IpXG4jIFRoZSBzdGF0ZSBjb2x1bW4gaGFzIHF1aXRlIHNvbWUgZHVwbGljYXRlczogXG4jIFwiQW5kYWx1Y1x1MDBlZGFcIiBhbmQgXCJBbmRhbHVzaWFcIixcIkFyYWdcdTAwZjNuXCIgYW5kIFwiQXJhZ29uXCIsXCJDYXN0aWxsYS1MYS1NYW5jaGFcIiBhbmQgXCJDYXN0aWxlLUxhIE1hbmNoYVwiLCBcIkNhc3RpbGxhIHkgTGVcdTAwZjNuXCIgYW5kIFwiQ2FzdGlsZSBhbmQgTGVcdTAwZjNuXCIsIFwiQ2F0YWx1XHUwMGYxYVwiIGFuZCBcIkNhdGFsb25pYVwiLCBcIkNvbXVuaWRhZCBkZSBNYWRyaWRcIiBhbmQgXCJDb21tdW5pdHkgb2YgTWFkcmlkXCIsIFwiQ29tdW5pZGFkIFZhbGVuY2lhbmFcIiBhbmQgXCJWYWxlbmNpYW4gQ29tbXVuaXR5XCIsIFwiUGFcdTAwZWRzIFZhc2NvXCIgYW5kIFwiQmFzcXVlIENvdW50cnlcIlxuIyBSZWNvZGUgYWxsIFNwYW5pc2ggbmFtZXMgdG8gdGhlaXIgRW5nbGlzaCBhbHRlcm5hdGl2ZVxuZGF0YVNldCRzdGF0ZTwtcmVjb2RlKGRhdGFTZXQkc3RhdGUsIFwiQW5kYWx1Y1x1MDBlZGFcIj1cIkFuZGFsdXNpYVwiICxcIkFyYWdcdTAwZjNuXCIgPSBcIkFyYWdvblwiLFwiQ2FzdGlsbGEtTGEtTWFuY2hhXCI9IFwiQ2FzdGlsZS1MYSBNYW5jaGFcIiwgXCJDYXN0aWxsYSB5IExlXHUwMGYzblwiPVwiQ2FzdGlsZS1hbmQtTGVcdTAwZjNuXCIsIFwiQ2F0YWx1XHUwMGYxYVwiPVwiQ2F0YWxvbmlhXCIsIFwiQ29tdW5pZGFkIGRlIE1hZHJpZFwiPVwiQ29tbXVuaXR5IG9mIE1hZHJpZFwiLCBcIkNvbXVuaWRhZCBWYWxlbmNpYW5hXCI9XCJWYWxlbmNpYW4gQ29tbXVuaXR5XCIsIFwiUGFcdTAwZWRzIFZhc2NvXCI9XCJCYXNxdWUgQ291bnRyeVwiKVxuIyBQcmludCB0aGUgbmV3IGxldmVscyBvZiB0aGUgYHN0YXRlYCBjb2x1bW5cbmxldmVscyhkYXRhU2V0JHN0YXRlKSIsInNjdCI6InRlc3RfbGlicmFyeV9mdW5jdGlvbihcImRwbHlyXCIsIGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gbG9hZCB0aGUgZHBseXIgcGFja2FnZSFcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKCdsZXZlbHMoZGF0YVNldCRzdGF0ZSknLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIHByaW50IHRoZSBuZXcgbGV2ZWxzIG9mIHRoZSBmYWN0b3IgJ3N0YXRlJyFcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

4.3 Re-organizing factors

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgUHJpbnQgaXRzIGxldmVsc1xuXG4jIFJlbGV2ZWwgdGhlIGNvcnB1cyBmYWN0b3I6IGl0cyBmaXJzdCB2YWx1ZSBzaG91bGQgYmUgJ1R3aXR0ZXInXG5cbiMgUHJpbnQgaXRzIGxldmVscyBhZ2FpbiIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBjb3JwdXMgdG8gYSBmYWN0b3JcbmRhdGFTZXQkY29ycHVzIDwtIGFzLmZhY3RvcihkYXRhU2V0JGNvcnB1cylcbiMgUHJpbnQgaXRzIGxldmVsc1xubGV2ZWxzKGRhdGFTZXQkY29ycHVzKVxuIyBSZWxldmVsIHRoZSBjb3JwdXMgZmFjdG9yOiBpdHMgZmlyc3QgdmFsdWUgc2hvdWxkIGJlICdUd2l0dGVyJ1xuZGF0YVNldCRjb3JwdXMgPC0gcmVsZXZlbChkYXRhU2V0JGNvcnB1cywgcmVmPVwiVHdpdHRlclwiKVxuIyBQcmludCBpdHMgbGV2ZWxzIGFnYWluXG5sZXZlbHMoZGF0YVNldCRjb3JwdXMpIiwic2N0IjoidGVzdF9vdXRwdXRfY29udGFpbnMoXCJsZXZlbHMoZGF0YVNldCRjb3JwdXMpXCIsIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgbGV2ZWxzIG9mIHRoZSBmYWN0b3IhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

5. Frequencies: counting values

5.1 Counting the negation variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gbmVnYXRpb24gdG8gYSBmYWN0b3JcblxuIyBDb21wdXRlIGEgdGFibGUgZm9yIHRoZSBuZWdhdGlvbiB2YXJpYWJsZSwgc3RvcmUgaXQgaW4gdGhlIHZhcmlhYmxlICduZWdhdGlvblRhYidcblxuIyBDb252ZXJ0ICduZWdhdGlvblRhYicgdG8gYSBkYXRhLmZyYW1lXG5cbiMgUHJpbnQgJ25lZ2F0aW9uVGFiJyIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBuZWdhdGlvbiB0byBhIGZhY3RvclxuZGF0YVNldCRuZWdhdGlvbjwtYXMuZmFjdG9yKGRhdGFTZXQkbmVnYXRpb24pXG4jIENvbXB1dGUgYSB0YWJsZSBmb3IgdGhlIG5lZ2F0aW9uIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ25lZ2F0aW9uVGFiJ1xubmVnYXRpb25UYWI8LXRhYmxlKGRhdGFTZXQkbmVnYXRpb24pXG4jIENvbnZlcnQgJ25lZ2F0aW9uVGFiJyB0byBhIGRhdGEuZnJhbWVcbm5lZ2F0aW9uVGFiPC1hcy5kYXRhLmZyYW1lKG5lZ2F0aW9uVGFiKVxuIyBQcmludCAnbmVnYXRpb25UYWInXG5uZWdhdGlvblRhYiIsInNjdCI6InRlc3Rfb2JqZWN0KFwibmVnYXRpb25UYWJcIixpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIGNvbXB1dGUgdGhlIHRhYmxlIVwiKVxudGVzdF9vdXRwdXRfY29udGFpbnMoXCJuZWdhdGlvblRhYlwiLCBpbmNvcnJlY3RfbXNnID0gXCJNYWtlIHN1cmUgdG8gcHJpbnQgdGhlIHRhYmxlIVwiKVxuc3VjY2Vzc19tc2coXCJHcmVhdCFcIikifQ==

5.2 Counting the province variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gcHJvdmluY2UgdG8gYSBmYWN0b3JcblxuIyBDb21wdXRlIGEgdGFibGUgZm9yIHRoZSBwcm92aW5jZSB2YXJpYWJsZSwgc3RvcmUgaXQgaW4gdGhlIHZhcmlhYmxlICdwcm92aW5jZVRhYidcblxuIyBDb252ZXJ0ICdwcm92aW5jZVRhYicgdG8gYSBkYXRhLmZyYW1lXG5cbiMgUHJpbnQgJ3Byb3ZpbmNlVGFiJyIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBwcm92aW5jZSB0byBhIGZhY3RvclxuZGF0YVNldCRwcm92aW5jZTwtYXMuZmFjdG9yKGRhdGFTZXQkcHJvdmluY2UpXG4jIENvbXB1dGUgYSB0YWJsZSBmb3IgdGhlIHByb3ZpbmNlIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ3Byb3ZpbmNlVGFiJ1xucHJvdmluY2VUYWI8LXRhYmxlKGRhdGFTZXQkcHJvdmluY2UpXG4jIENvbnZlcnQgJ3Byb3ZpbmNlVGFiJyB0byBhIGRhdGEuZnJhbWVcbnByb3ZpbmNlVGFiPC1hcy5kYXRhLmZyYW1lKHByb3ZpbmNlVGFiKVxuIyBQcmludCAncHJvdmluY2VUYWInXG5wcm92aW5jZVRhYiIsInNjdCI6InRlc3Rfb2JqZWN0KFwicHJvdmluY2VUYWJcIiwgaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBjb21wdXRlIHRoZSB0YWJsZSFcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKFwicHJvdmluY2VUYWJcIiwgaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgdGFibGUhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

5.3 Counting the Typical.Action.Chain.Pos variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHRvIGEgZmFjdG9yXG5cbiMgQ29tcHV0ZSBhIHRhYmxlIGZvciB0aGUgVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYidcblxuIyBDb252ZXJ0ICdUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWInIHRvIGEgZGF0YS5mcmFtZVxuXG4jIFByaW50ICdUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWInIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG5cbiMgQ29udmVydCB0aGUgY29sdW1uIFR5cGljYWwuQWN0aW9uLkNoYWluLlBvcyB0byBhIGZhY3RvclxuZGF0YVNldCRUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3M8LWFzLmZhY3RvcihkYXRhU2V0JFR5cGljYWwuQWN0aW9uLkNoYWluLlBvcylcbiMgQ29tcHV0ZSBhIHRhYmxlIGZvciB0aGUgVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYidcblR5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYjwtdGFibGUoZGF0YVNldCRUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3MpXG4jIENvbnZlcnQgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYicgdG8gYSBkYXRhLmZyYW1lXG5UeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWI8LWFzLmRhdGEuZnJhbWUoVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiKVxuIyBQcmludCAnVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiJ1xuVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiIiwic2N0IjoidGVzdF9vYmplY3QoXCJUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWJcIixpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIGNvbXB1dGUgdGhlIHRhYmxlIVwiKVxudGVzdF9vdXRwdXRfY29udGFpbnMoXCJUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWJcIiwgaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgdGFibGUhXCIpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

6. Proportions

6.1 Proportions for the Typical.Action.Chain.Pos variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHRvIGEgZmFjdG9yXG5cbiMgQ29tcHV0ZSBhIHRhYmxlIGZvciB0aGUgVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYidcblxuIyBDb21wdXRlIHByb3BvcnRpb25zIGZvciB0aGF0IHRhYmxlXG5cbiMgUHJpbnQgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYiciLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zIHRvIGEgZmFjdG9yXG5kYXRhU2V0JFR5cGljYWwuQWN0aW9uLkNoYWluLlBvczwtYXMuZmFjdG9yKGRhdGFTZXQkVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zKVxuIyBDb21wdXRlIGEgdGFibGUgZm9yIHRoZSBUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3MgdmFyaWFibGUsIHN0b3JlIGl0IGluIHRoZSB2YXJpYWJsZSAnVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiJ1xuVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiPC10YWJsZShkYXRhU2V0JFR5cGljYWwuQWN0aW9uLkNoYWluLlBvcylcbiMgQ29tcHV0ZSBwcm9wb3J0aW9ucyBmb3IgdGhhdCB0YWJsZVxuVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiPC1wcm9wLnRhYmxlKFR5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYilcbiMgUHJpbnQgJ1R5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYidcblR5cGljYWwuQWN0aW9uLkNoYWluLlBvc1RhYiIsInNjdCI6InRlc3Rfb2JqZWN0KFwiVHlwaWNhbC5BY3Rpb24uQ2hhaW4uUG9zVGFiXCIsaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBjb21wdXRlIHRoZSB0YWJsZSFcIilcbnRlc3Rfb3V0cHV0X2NvbnRhaW5zKCdUeXBpY2FsLkFjdGlvbi5DaGFpbi5Qb3NUYWInLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIGNvbXB1dGUgcHJvcG9ydGlvbnMgZm9yIHRoZSB0YWJsZSFcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

6.2 Proportions for the corpus variable

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgQ29tcHV0ZSBhIHRhYmxlIGZvciB0aGUgY29ycHVzIHZhcmlhYmxlLCBzdG9yZSBpdCBpbiB0aGUgdmFyaWFibGUgJ2NvcnB1c1RhYidcblxuIyBDb252ZXJ0IHRoZSBjb3JwdXNUYWIgb2JqZWN0IHRvIGEgZGF0YS5mcmFtZSBcblxuIyBDb21wdXRlIHByb3BvcnRpb25zIGJhc2VkIG9uIHRoZSBGcmVxIGNvbHVtbiBvZiB0aGF0IGRhdGEuZnJhbWUuIFN0b3JlIHRoZSBwcm9wb3J0aW9ucyBpbiBhIG5ldyBjb2x1bW4gY2FsbGVkICdwcm9wJ1xuXG4jIFByaW50IGNvcnB1c1RhYiIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBjb3JwdXMgdG8gYSBmYWN0b3JcbmRhdGFTZXQkY29ycHVzPC1hcy5mYWN0b3IoZGF0YVNldCRjb3JwdXMpXG4jIENvbXB1dGUgYSB0YWJsZSBmb3IgdGhlIGNvcnB1cyB2YXJpYWJsZSwgc3RvcmUgaXQgaW4gdGhlIHZhcmlhYmxlICdjb3JwdXNUYWInXG5jb3JwdXNUYWI8LXRhYmxlKGRhdGFTZXQkY29ycHVzKVxuIyBDb252ZXJ0ICdjb3JwdXNUYWInIHRvIGEgZGF0YS5mcmFtZVxuY29ycHVzVGFiPC1hcy5kYXRhLmZyYW1lKGNvcnB1c1RhYilcbiMgQ29tcHV0ZSBwcm9wb3J0aW9ucyBiYXNlZCBvbiB0aGUgRnJlcSBjb2x1bW4gb2YgdGhhdCBkYXRhLmZyYW1lLiBTdG9yZSB0aGUgcHJvcG9ydGlvbnNpbiBhIG5ldyBjb2x1bW4gY2FsbGVkICdwcm9wJ1xuY29ycHVzVGFiJHByb3A8LWNvcnB1c1RhYiRGcmVxL3N1bShjb3JwdXNUYWIkRnJlcSwgbmEucm09VFJVRSlcbiMgUHJpbnQgY29ycHVzVGFiXG5jb3JwdXNUYWIiLCJzY3QiOiJ0ZXN0X29iamVjdChcImNvcnB1c1RhYlwiLGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gY29tcHV0ZSB0aGUgdGFibGUhXCIpXG50ZXN0X291dHB1dF9jb250YWlucygnY29ycHVzVGFiJywgIGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gY29tcHV0ZSBwcm9wb3J0aW9ucyBmb3IgdGhlIHRhYmxlIVwiKVxuc3VjY2Vzc19tc2coXCJHcmVhdCFcIikifQ==

7. Bar plots

7.1 Exploring the counts of the corpus variable (1/2)

  • ANALYZE:
  • Which corpus provided the most examples?
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgUHJpbnQgYSB0YWJsZSBvZiB0aGUgY29ycHVzIGZhY3RvciAiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5kYXRhU2V0JGNvcnB1czwtYXMuZmFjdG9yKGRhdGFTZXQkY29ycHVzKVxuXG4jIFByaW50IGEgdGFibGUgb2YgdGhlIGNvcnB1cyBmYWN0b3IgXG50YWJsZShkYXRhU2V0JGNvcnB1cykiLCJzY3QiOiJ0ZXN0X291dHB1dF9jb250YWlucyhcInRhYmxlKGRhdGFTZXQkY29ycHVzKVwiLCBpbmNvcnJlY3RfbXNnID0gXCJEb24ndCBmb3JnZXQgdG8gcHJpbnQgdGhlIHRhYmxlIVwiKVxuc3VjY2Vzc19tc2coXCJHcmVhdCFcIikifQ==

7.1 Exploring the counts of the corpus variable (2/2)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgTG9hZCBnZ3Bsb3QyXG5cbiMgRHJhdyBhIGJhc2ljIGJhciBjaGFydCBvZiB0aGUgY29ycHVzIGZhY3RvciIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBjb3JwdXMgdG8gYSBmYWN0b3JcbmRhdGFTZXQkY29ycHVzPC1hcy5mYWN0b3IoZGF0YVNldCRjb3JwdXMpXG4jIExvYWQgZ2dwbG90MlxubGlicmFyeShnZ3Bsb3QyKVxuIyBEcmF3IGEgYmFzaWMgYmFyIGNoYXJ0IG9mIHRoZSBjb3JwdXMgZmFjdG9yXG5nZ3Bsb3QoZGF0YVNldCwgYWVzKHg9Y29ycHVzKSkgKyBnZW9tX2JhcigpIiwic2N0IjoidGVzdF9saWJyYXJ5X2Z1bmN0aW9uKFwiZ2dwbG90MlwiLCBpbmNvcnJlY3RfbXNnID0gXCJEb24ndCBmb3JnZXQgdG8gbG9hZCBnZ3Bsb3QyXCIpXG50ZXN0X2dncGxvdChpbmRleCA9IDEpXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

7.2 Exploring the counts of the broad.regions variable (1/2)

  • ANALYZE:
  • Which broad.regions provided the most examples?
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gYnJvYWQucmVnaW9ucyB0byBhIGZhY3RvclxuXG4jIFByaW50IGEgdGFibGUgb2YgdGhlIHByb3BvcnRpb25zIG9mIHRoZSBicm9hZC5yZWdpb25zIGZhY3RvciAiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gYnJvYWQucmVnaW9ucyB0byBhIGZhY3RvclxuZGF0YVNldCRicm9hZC5yZWdpb25zPC1hcy5mYWN0b3IoZGF0YVNldCRicm9hZC5yZWdpb25zKVxuXG4jIFByaW50IGEgdGFibGUgb2YgdGhlIHByb3BvcnRpb25zIG9mIHRoZSBicm9hZC5yZWdpb25zIGZhY3RvciBcbnRhYmxlKGRhdGFTZXQkYnJvYWQucmVnaW9ucykiLCJzY3QiOiIgdGVzdF9vdXRwdXRfY29udGFpbnMoJ3RhYmxlKGRhdGFTZXQkYnJvYWQucmVnaW9ucyknLCAgaW5jb3JyZWN0X21zZyA9IFwiTWFrZSBzdXJlIHRvIHByaW50IHRoZSB0YWJsZSFcIilcblxuc3VjY2Vzc19tc2coXCJHcmVhdCFcIikifQ==

7.2 Exploring the counts of the broad.regions variable (2/2)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gYnJvYWQucmVnaW9ucyB0byBhIGZhY3RvclxuXG4jIExvYWQgZ2dwbG90MlxuXG4jIERyYXcgYSBwcmV0dHkgYmFyIGNoYXJ0IG9mIHRoZSBicm9hZC5yZWdpb25zIGZhY3RvcjogZWFjaCByZWdpb24gaGFzIGl0cyBvd24gZmlsbCBhbmQgY29sb3IsIHRoZXJlIGlzIG5vIGxlZ2VuZCB0aXRsZSwgdGhlIFkgYXhpcyB0aXRsZSBpcyBcIk51bWJlciBvZiBvY2N1cnJlbmNlc1wiLCB0aGUgWCBheGlzIHRpdGxlIGlzIFwiTGFyZ2UgcmVnaW9uc1wiLCB0aGUgcGxvdCB0aXRsZSBpcyBcIk51bWJlciBvZiBvY2N1cnJlbmNlcyBwZXIgcmVnaW9uXCIiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gYnJvYWQucmVnaW9ucyB0byBhIGZhY3RvclxuZGF0YVNldCRicm9hZC5yZWdpb25zPC1hcy5mYWN0b3IoZGF0YVNldCRicm9hZC5yZWdpb25zKVxuIyBMb2FkIGdncGxvdDJcbmxpYnJhcnkoZ2dwbG90MilcbiMgRHJhdyBhIGJhc2ljIGJhciBjaGFydCBvZiB0aGUgYnJvYWQucmVnaW9ucyBmYWN0b3JcbmdncGxvdChkYXRhU2V0LCBhZXMoeD1icm9hZC5yZWdpb25zLCBmaWxsPWJyb2FkLnJlZ2lvbnMsIGNvbG9yPWJyb2FkLnJlZ2lvbnMpKSArIGdlb21fYmFyKCkgKyB0aGVtZShsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCkpICsgbGFicyh4PVwiTGFyZ2UgcmVnaW9uc1wiLCB5PVwiTnVtYmVyIG9mIG9jY3VycmVuY2VzXCIsIHRpdGxlPVwiTnVtYmVyIG9mIG9jY3VycmVuY2VzIHBlciByZWdpb25cIikiLCJzY3QiOiJ0ZXN0X2xpYnJhcnlfZnVuY3Rpb24oXCJnZ3Bsb3QyXCIpXG50ZXN0X2dncGxvdChpbmRleD0xLCBhbGxfZmFpbF9tc2cgPSBcIkNvbXB1dGVyIHNheXMgbm8uIFRoaXMgcGxvdCBpcyBjb21wbGV0ZWx5IHdyb25nXCIsIGRhdGFfZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgaW5jbHVkZSB0aGUgZGF0YSBpbiB0aGUgZ2dwbG90IGNhbGxcIiwgYWVzX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGRlZmluZSB0aGUgY29ycmVjdCBhZXN0aGV0aWNzIChhZXMpXCIsIGdlb21fZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgZGVmaW5lIHRoZSBjb3JyZWN0IHBsb3R0aW5nIGZ1bmN0aW9uIChnZW9tXylcIilcbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

7.3 Exploring the the counts of the negation variable

  • ANALYZE:
    • Do tokens with and without negation occur with equal frequency?
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gbmVnYXRpb24gdG8gYSBmYWN0b3JcblxuIyBQcmludCBhIHRhYmxlIG9mIHRoZSBuZWdhdGlvbiBjb2x1bW5cblxuIyBMb2FkIGdncGxvdDJcblxuIyBEcmF3IGEgcHJldHR5IGJhciBjaGFydCBvZiB0aGUgbmVnYXRpb24gZmFjdG9yOiBlYWNoIHZhbHVlIGhhcyBpdHMgb3duIGZpbGwgYW5kIGNvbG9yLCB0aGVyZSBpcyBubyBsZWdlbmQgdGl0bGUsIHRoZSBZIGF4aXMgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXNcIiwgdGhlIFggYXhpcyB0aXRsZSBpcyBcIkFic2VuY2UvcHJlc2VuY2Ugb2YgbmVnYXRpb25cIiwgdGhlIHBsb3QgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXMgcGVyIEFic2VuY2UvcHJlc2VuY2Ugb2YgbmVnYXRpb25cIiIsInNvbHV0aW9uIjoiIyBUaGUgZGF0YS5mcmFtZSBkYXRhU2V0IGlzIGFscmVhZHkgaW4geW91ciB3b3Jrc3BhY2UuIFxuXG4jIENvbnZlcnQgdGhlIGNvbHVtbiBuZWdhdGlvbiB0byBhIGZhY3RvclxuZGF0YVNldCRuZWdhdGlvbjwtYXMuZmFjdG9yKGRhdGFTZXQkbmVnYXRpb24pXG4jIFByaW50IGEgdGFibGUgb2YgdGhlIG5lZ2F0aW9uIGNvbHVtblxudGFibGUoZGF0YVNldCRuZWdhdGlvbilcbiMgTG9hZCBnZ3Bsb3QyXG5saWJyYXJ5KGdncGxvdDIpXG4jIERyYXcgYSBiYXNpYyBiYXIgY2hhcnQgb2YgdGhlIG5lZ2F0aW9uIGZhY3RvclxuZ2dwbG90KGRhdGFTZXQsIGFlcyh4PW5lZ2F0aW9uLCBmaWxsPW5lZ2F0aW9uLCBjb2xvcj1uZWdhdGlvbikpICsgZ2VvbV9iYXIoKSArIHRoZW1lKGxlZ2VuZC50aXRsZSA9IGVsZW1lbnRfYmxhbmsoKSkgKyBsYWJzKHg9XCJMYXJnZSByZWdpb25zXCIsIHk9XCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXNcIiwgdGl0bGU9XCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXMgcGVyIHJlZ2lvblwiKSIsInNjdCI6InRlc3Rfb3V0cHV0X2NvbnRhaW5zKFwidGFibGUoZGF0YVNldCRuZWdhdGlvblwiLCBpbmNvcnJlY3RfbXNnID0gXCJNYWtlIHN1cmUgdG8gcHJpbnQgYSB0YWJsZSBvZiBuZWdhdGlvblwiKVxudGVzdF9saWJyYXJ5X2Z1bmN0aW9uKFwiZ2dwbG90MlwiKVxudGVzdF9nZ3Bsb3QoaW5kZXg9MSwgYWxsX2ZhaWxfbXNnID0gXCJDb21wdXRlciBzYXlzIG5vLiBUaGlzIHBsb3QgaXMgY29tcGxldGVseSB3cm9uZ1wiLCBkYXRhX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGluY2x1ZGUgdGhlIGRhdGEgaW4gdGhlIGdncGxvdCBjYWxsXCIsIGFlc19mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBkZWZpbmUgdGhlIGNvcnJlY3QgYWVzdGhldGljcyAoYWVzKVwiLCBnZW9tX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGRlZmluZSB0aGUgY29ycmVjdCBwbG90dGluZyBmdW5jdGlvbiAoZ2VvbV8pXCIpXG5cbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

7.4 Exploring the proportions of the corpus variable (1/2)

  • ANALYZE:
    • Which corpus provided the most examples?
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgUHJpbnQgYSB0YWJsZSBvZiB0aGUgcHJvcG9ydGlvbnMgb2YgdGhlIGNvcnB1cyBmYWN0b3IgIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG5cbiMgQ29udmVydCB0aGUgY29sdW1uIGNvcnB1cyB0byBhIGZhY3RvclxuZGF0YVNldCRjb3JwdXM8LWFzLmZhY3RvcihkYXRhU2V0JGNvcnB1cylcblxuIyBQcmludCBhIHRhYmxlIG9mIHRoZSBwcm9wb3J0aW9ucyBvZiB0aGUgY29ycHVzIGZhY3RvciBcbnByb3AudGFibGUodGFibGUoZGF0YVNldCRjb3JwdXMpKSIsInNjdCI6IiB0ZXN0X291dHB1dF9jb250YWlucygncHJvcC50YWJsZSh0YWJsZShkYXRhU2V0JGNvcnB1cykpJywgIGluY29ycmVjdF9tc2cgPSBcIk1ha2Ugc3VyZSB0byBwcmludCB0aGUgdGFibGUhXCIpXG5cbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

7.4 Exploring the proportions of the corpus variable (2/2)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5cbiMgTG9hZCBnZ3Bsb3QyXG5cbiMgRHJhdyBhIGJhc2ljIGJhciBjaGFydCBvZiB0aGUgcHJvcG9ydGlvbnMgb2YgdGhlIGNvcnB1cyBmYWN0b3IiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gY29ycHVzIHRvIGEgZmFjdG9yXG5kYXRhU2V0JGNvcnB1czwtYXMuZmFjdG9yKGRhdGFTZXQkY29ycHVzKVxuIyBMb2FkIGdncGxvdDJcbmxpYnJhcnkoZ2dwbG90MilcbiMgRHJhdyBhIGJhc2ljIGJhciBjaGFydCBvZiB0aGUgcHJvcG9ydGlvbnMgb2YgdGhlIGNvcnB1cyBmYWN0b3JcbmdncGxvdChkYXRhU2V0LCBhZXMoeD1jb3JwdXMpKSArIGdlb21fYmFyKGFlcyh5PS4uY291bnQuLi9zdW0oLi5jb3VudC4uKSkpIiwic2N0IjoidGVzdF9saWJyYXJ5X2Z1bmN0aW9uKFwiZ2dwbG90MlwiKVxudGVzdF9nZ3Bsb3QoaW5kZXg9MSwgYWxsX2ZhaWxfbXNnID0gXCJDb21wdXRlciBzYXlzIG5vLiBUaGlzIHBsb3QgaXMgY29tcGxldGVseSB3cm9uZ1wiLCBkYXRhX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGluY2x1ZGUgdGhlIGRhdGEgaW4gdGhlIGdncGxvdCBjYWxsXCIsIGFlc19mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBkZWZpbmUgdGhlIGNvcnJlY3QgYWVzdGhldGljcyAoYWVzKVwiLCBnZW9tX2ZhaWxfbXNnID0gXCJNYWtlIHN1cmUgeW91IGRlZmluZSB0aGUgY29ycmVjdCBwbG90dGluZyBmdW5jdGlvbiAoZ2VvbV8pXCIpXG5cbnN1Y2Nlc3NfbXNnKFwiR3JlYXQhXCIpIn0=

7.5 Exploring the proportions of the broad.regions variable (1/2)

  • ANALYZE:
    • Which broad.regions provided the most examples?

7.5 Exploring the proportions of the broad.regions variable (2/2)

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gYnJvYWQucmVnaW9ucyB0byBhIGZhY3RvclxuXG4jIExvYWQgZ2dwbG90MlxuXG4jIERyYXcgYSBwcmV0dHkgYmFyIGNoYXJ0IG9mIHRoZSBwcm9wb3J0aW9ucyBvZiB0aGUgYnJvYWQucmVnaW9ucyBmYWN0b3I6IGVhY2ggcmVnaW9uIGhhcyBpdHMgb3duIGZpbGwgYW5kIGNvbG9yLCB0aGVyZSBpcyBubyBsZWdlbmQgdGl0bGUsIHRoZSBZIGF4aXMgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXNcIiwgdGhlIFggYXhpcyB0aXRsZSBpcyBcIkxhcmdlIHJlZ2lvbnNcIiwgdGhlIHBsb3QgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXMgcGVyIHJlZ2lvblwiIiwic29sdXRpb24iOiIjIFRoZSBkYXRhLmZyYW1lIGRhdGFTZXQgaXMgYWxyZWFkeSBpbiB5b3VyIHdvcmtzcGFjZS4gXG5cbiMgQ29udmVydCB0aGUgY29sdW1uIGJyb2FkLnJlZ2lvbnMgdG8gYSBmYWN0b3JcbmRhdGFTZXQkYnJvYWQucmVnaW9uczwtYXMuZmFjdG9yKGRhdGFTZXQkYnJvYWQucmVnaW9ucylcbiMgTG9hZCBnZ3Bsb3QyXG5saWJyYXJ5KGdncGxvdDIpXG4jIERyYXcgYSBwcmV0dHkgYmFyIGNoYXJ0IG9mIHRoZSBwcm9wb3J0aW9ucyBvZiB0aGUgYnJvYWQucmVnaW9ucyBmYWN0b3I6IGVhY2ggcmVnaW9uIGhhcyBpdHMgb3duIGZpbGwgYW5kIGNvbG9yLCB0aGVyZSBpcyBubyBsZWdlbmQgdGl0bGUsIHRoZSBZIGF4aXMgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXNcIiwgdGhlIFggYXhpcyB0aXRsZSBpcyBcIkxhcmdlIHJlZ2lvbnNcIiwgdGhlIHBsb3QgdGl0bGUgaXMgXCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXMgcGVyIHJlZ2lvblwiXG5nZ3Bsb3QoZGF0YVNldCwgYWVzKHg9YnJvYWQucmVnaW9ucywgZmlsbD1icm9hZC5yZWdpb25zLCBjb2xvcj1icm9hZC5yZWdpb25zKSkgKyBnZW9tX2JhcihhZXMoeT0uLmNvdW50Li4vc3VtKC4uY291bnQuLikpKSArIHRoZW1lKGxlZ2VuZC50aXRsZSA9IGVsZW1lbnRfYmxhbmsoKSkgKyBsYWJzKHg9XCJMYXJnZSByZWdpb25zXCIsIHk9XCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXNcIiwgdGl0bGU9XCJOdW1iZXIgb2Ygb2NjdXJyZW5jZXMgcGVyIHJlZ2lvblwiKSIsInNjdCI6InRlc3RfbGlicmFyeV9mdW5jdGlvbihcImdncGxvdDJcIilcbnRlc3RfZ2dwbG90KGluZGV4PTEsIGFsbF9mYWlsX21zZyA9IFwiQ29tcHV0ZXIgc2F5cyBuby4gVGhpcyBwbG90IGlzIGNvbXBsZXRlbHkgd3JvbmdcIiwgZGF0YV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBpbmNsdWRlIHRoZSBkYXRhIGluIHRoZSBnZ3Bsb3QgY2FsbFwiLCBhZXNfZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgZGVmaW5lIHRoZSBjb3JyZWN0IGFlc3RoZXRpY3MgKGFlcylcIiwgZ2VvbV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBkZWZpbmUgdGhlIGNvcnJlY3QgcGxvdHRpbmcgZnVuY3Rpb24gKGdlb21fKVwiKVxuc3VjY2Vzc19tc2coXCJHcmVhdCFcIikifQ==

7.6 Exploring the the proportions of the negation variable

  • ANALYZE: - Do tokens with and without negation occur with equal frequency?
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IlN5cy5zZXRsb2NhbGUoY2F0ZWdvcnkgPSAnTENfQUxMJywnZW5fVVMuVVRGLTgnKVxubGlicmFyeShyZWFkcilcbmRhdGFTZXQgPC0gcmVhZF9jc3YoXCJodHRwOi8vd3d3Lmplcm9lbmNsYWVzLmJlL3N0YXRpc3RpY3NfZm9yX2xpbmd1aXN0aWNzL2RhdGFzZXRzL2NsYXNzM19jbGFlc18yMDE3LmNzdlwiKSIsInNhbXBsZSI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gbmVnYXRpb24gdG8gYSBmYWN0b3JcblxuIyBQcmludCBhIHRhYmxlIG9mIHRoZSBuZWdhdGlvbiBjb2x1bW5cblxuIyBMb2FkIGdncGxvdDJcblxuIyBEcmF3IGEgcHJldHR5IGJhciBjaGFydCBvZiB0aGUgcHJvcG9ydGlvbnMgb2YgdGhlIG5lZ2F0aW9uIGZhY3RvcjogZWFjaCB2YWx1ZSBoYXMgaXRzIG93biBmaWxsIGFuZCBjb2xvciwgdGhlcmUgaXMgbm8gbGVnZW5kIHRpdGxlLCB0aGUgWSBheGlzIHRpdGxlIGlzIFwiTnVtYmVyIG9mIG9jY3VycmVuY2VzXCIsIHRoZSBYIGF4aXMgdGl0bGUgaXMgXCJBYnNlbmNlL3ByZXNlbmNlIG9mIG5lZ2F0aW9uXCIsIHRoZSBwbG90IHRpdGxlIGlzIFwiTnVtYmVyIG9mIG9jY3VycmVuY2VzIHBlciBBYnNlbmNlL3ByZXNlbmNlIG9mIG5lZ2F0aW9uXCIiLCJzb2x1dGlvbiI6IiMgVGhlIGRhdGEuZnJhbWUgZGF0YVNldCBpcyBhbHJlYWR5IGluIHlvdXIgd29ya3NwYWNlLiBcblxuIyBDb252ZXJ0IHRoZSBjb2x1bW4gbmVnYXRpb24gdG8gYSBmYWN0b3JcbmRhdGFTZXQkbmVnYXRpb248LWFzLmZhY3RvcihkYXRhU2V0JG5lZ2F0aW9uKVxuIyBQcmludCBhIHRhYmxlIG9mIHRoZSBuZWdhdGlvbiBjb2x1bW5cbnRhYmxlKGRhdGFTZXQkbmVnYXRpb24pXG4jIExvYWQgZ2dwbG90MlxubGlicmFyeShnZ3Bsb3QyKVxuIyBEcmF3IGEgcHJldHR5IGJhciBjaGFydCBvZiB0aGUgcHJvcG9ydGlvbnMgb2YgdGhlIG5lZ2F0aW9uIGZhY3RvcjogZWFjaCB2YWx1ZSBoYXMgaXRzIG93biBmaWxsIGFuZCBjb2xvciwgdGhlcmUgaXMgbm8gbGVnZW5kIHRpdGxlLCB0aGUgWSBheGlzIHRpdGxlIGlzIFwiTnVtYmVyIG9mIG9jY3VycmVuY2VzXCIsIHRoZSBYIGF4aXMgdGl0bGUgaXMgXCJBYnNlbmNlL3ByZXNlbmNlIG9mIG5lZ2F0aW9uXCIsIHRoZSBwbG90IHRpdGxlIGlzIFwiTnVtYmVyIG9mIG9jY3VycmVuY2VzIHBlciBBYnNlbmNlL3ByZXNlbmNlIG9mIG5lZ2F0aW9uXCJcbmdncGxvdChkYXRhU2V0LCBhZXMoeD1uZWdhdGlvbiwgZmlsbD1uZWdhdGlvbiwgY29sb3I9bmVnYXRpb24pKSArIGdlb21fYmFyKGFlcyh5PS4uY291bnQuLi9zdW0oLi5jb3VudC4uKSkpICsgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKSArIGxhYnMoeD1cIkxhcmdlIHJlZ2lvbnNcIiwgeT1cIk51bWJlciBvZiBvY2N1cnJlbmNlc1wiLCB0aXRsZT1cIk51bWJlciBvZiBvY2N1cnJlbmNlcyBwZXIgcmVnaW9uXCIpIiwic2N0IjoidGVzdF9vdXRwdXRfY29udGFpbnMoXCJ0YWJsZShkYXRhU2V0JG5lZ2F0aW9uKVwiLCBcIlJlbWVtYmVyIHRvIHByaW50IGEgdGFibGUgb2YgbmVnYXRpb25cIilcbnRlc3RfbGlicmFyeV9mdW5jdGlvbihcImdncGxvdDJcIilcbnRlc3RfZ2dwbG90KGluZGV4PTEsIGFsbF9mYWlsX21zZyA9IFwiQ29tcHV0ZXIgc2F5cyBuby4gVGhpcyBwbG90IGlzIGNvbXBsZXRlbHkgd3JvbmdcIiwgZGF0YV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBpbmNsdWRlIHRoZSBkYXRhIGluIHRoZSBnZ3Bsb3QgY2FsbFwiLCBhZXNfZmFpbF9tc2cgPSBcIk1ha2Ugc3VyZSB5b3UgZGVmaW5lIHRoZSBjb3JyZWN0IGFlc3RoZXRpY3MgKGFlcylcIiwgZ2VvbV9mYWlsX21zZyA9IFwiTWFrZSBzdXJlIHlvdSBkZWZpbmUgdGhlIGNvcnJlY3QgcGxvdHRpbmcgZnVuY3Rpb24gKGdlb21fKVwiKVxuXG5zdWNjZXNzX21zZyhcIkdyZWF0IVwiKSJ9

8. References

  • Claes, J. (2017). Cognitive and geographic constraints on morphosyntactic variation: The variable agreement of presentational haber in Peninsular Spanish. Belgian Journal of Linguistics (31), 28-53.

© 2018 Jeroen Claes