четверг, 15 октября 2020 г.

[Links for myself] Clojure book, Machine Learning, preschool, free GPUs

Well, old good times - if you so old as me you should be familiar with lists of favorite links.

(Btw, on a picture - pub in my town, beer & rock'n'roll included)

Clojure web development book (HTTP, Routing , Middleware): https://grishaev.me/clj-book-web-1/ 

Writing Clojure web applications with RING (library, not a framework): https://www.baeldung.com/clojure-ring

Hitchhikers guide to Machine Learning (algorithms pros and cons, explanation like for school kids): https://tproger.ru/translations/hitchhikers-guide-to-ml/

Google's Colab (free GPU, notebook style): http://colab.research.google.com/

Kaggle (a lot of datasets for hardware, good to play with monitoring data, free GPU hours): https://www.kaggle.com

UML diagrams I used for AWS architecture trainings and classes: https://www.lucidchart.com

Fun explanation about quantum effects (and how/why quantum computing works) https://www.youtube.com/watch?v=g_IaVepNDT4

IBM quantum experience (and lego bricks to play with: https://quantum-computing.ibm.com

Quantum development kit (by M$, emulator (I wondering how they emulate frozen qubits on my hot laptop) included): https://www.microsoft.com/en-us/quantum/development-kit and simulator itself https://github.com/StationQ/Liquid

And Qiskit: https://qiskit.org/

Oh, The Lord, please give me power to read all of it and the calmness and eggs of steel to understand 10% of it.

Still alive? Wiki: https://en.wikipedia.org/wiki/Cloud-based_quantum_computing


четверг, 1 октября 2020 г.

Data streaming 101: AWS Kinesis, Redshift, MySQL, Athena

These are my notes from a discovery task I have had:
  • EC2 can write to Kinesis data stream and then Firehose delivery stream can transform data with Lambda and store result at S3 or EMR.
  • EMR is good choice to run Spark application (Python, Java, Scala) and may use S3 as a destination storage.
  • Athena performs 30% better with S3 data formatted as Parquet compared to JSON.
  • Redshift is read-optimized while MySQL is write-optimized. 
  • Application performance with MySQL would benefit in loading small volumes of data more frequently. 
  • Redshift is more efficient at loading large volumes of data less frequently.
And a great video with Vladimir and Benjamin discussing evolution of architecture: