Tuesday, December 27, 2016

Big Data - Part 5 - AWS Athena

at

Monday, December 26, 2016

AWS Athena - ELB Log Analaysis



Thoughts


Able to get working Athena on our production logs.

No more ELK Stack, Hadoop , Map reduce, Big Query.


Create database apilogs2

CREATE EXTERNAL TABLE IF NOT EXISTS apilogs2.elb_logs_raw_native (
  request_timestamp string,
  elb_name string,
  request_ip string,
  request_port int,
  backend_ip string,
  backend_port int,
  request_processing_time double,
  backend_processing_time double,
  client_response_time double,
  elb_response_code string,
  backend_response_code string,
  received_bytes bigint,
  sent_bytes bigint,
  request_verb string,
  url string,
  protocol string,
  user_agent string,
  ssl_cipher string,
  ssl_protocol string )
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'

WITH SERDEPROPERTIES (
         'serialization.format' = '1','input.regex' = '([^ ]*) ([^ ]*) ([^ ]*):([0-9]*) ([^ ]*):([0-9]*) ([.0-9]*) ([.0-9]*) ([.0-9]*) (-|[0-9]*) (-|[0-9]*) ([-0-9]*) ([-0-9]*) \\\"([^ ]*) ([^ ]*) (- |[^ ]*)\\\" (\"[^\"]*\") ([A-Z0-9-]+) ([A-Za-z0-9.-]*)$' )

LOCATION 's3://....';


SELECT * FROM apilogs2.elb_logs_raw_native WHERE elb_response_code = '200' LIMIT 100;


SELECT * FROM apilogs2.elb_logs_raw_native WHERE elb_response_code = '200' and url like '%serialNumber%' LIMIT 100;


SELECT * FROM apilogs2.elb_logs_raw_native WHERE elb_response_code = '200'
and url like '%serialNumber%'
and user_agent like '%ok%' LIMIT 100; --18

at
Labels: ,

Big Data - Part 4



http://www.slideshare.net/AmazonWebServices/bdt310-big-data-architectural-patterns-and-best-practices-on-aws








Server less Architecture 


https://docs.aws.amazon.com/athena/latest/ug/json.html

https://medium.com/@pahud/kinesis-firehose-put-record-with-cli-e835a6e6ffe2#.j5t0okqux

http://aws.mannem.me/?p=1152



SnowPlow


http://snowplowanalytics.com/blog/2014/05/13/introducing-schemaver-for-semantic-versioning-of-schemas/

https://github.com/snowplow/snowplow

https://github.com/snowplow/snowplow/wiki/Setting-up-the-Cloudfront-collector

https://github.com/snowplow/snowplow/wiki/snowplow-tracker-protocol

http://www.slideshare.net/robkingston/a-taste-of-snowplow-analytics-data


Aws Lamda & Gateway


https://abhishek-tiwari.com/post/aws-lambda-and-endless-serverless-possibilities

 https://gist.github.com/markusklems/1e7218d76d7583f1f7b3
https://claudiajs.com/tutorials/index.html


https://www.youtube.com/channel/UClLLJjpSWRRa1BosQrNVDjA
at
Labels: ,

Sunday, December 25, 2016

Completed Serverless Architecture course





https://www.udemy.com/serverless-architecture-on-aws/learn/v4/overview




at
Labels: ,

AWS Lamda Deep Dive - Wrapping up the Course

at
Labels: ,

Thursday, December 8, 2016

fonts



https://creativemarket.com/blog/whats-the-difference-between-leading-kerning-and-tracking
http://old.fontlab.com/content/blogcategory/40/364/

https://www.youtube.com/watch?v=csCjP63IuSs&feature=share
http://www.1001fonts.com/web-fonts.html
https://superdevresources.com/create-font-from-handwriting/

https://bond.co/

https://github.com/dok/awesome-text-editing


https://helpx.adobe.com/animate/using/text-layout-framework-tlf-text.html



Text Layout Framework Technology Center


https://helpx.adobe.com/animate/using/text-layout-framework-tlf-text.html
http://www.adobe.com/devnet-apps/tlf/demo/
http://www.adobe.com/devnet/tlf.html
https://www.cocoanetics.com/2012/02/dtrichtexteditor-dtcoretext-news/
https://www.objc.io/issues/5-ios7/getting-to-know-textkit/
https://www.raywenderlich.com/4147/core-text-tutorial-for-ios-making-a-magazine-app






https://www.youtube.com/watch?v=RAsd8DzgtO0

https://www.youtube.com/watch?v=JeJZPAOFsJc

https://www.youtube.com/watch?v=YrLRku_zzIY

https://www.youtube.com/watch?v=YrLRku_zzIY




at
Subscribe to: Posts (Atom)