Monday, September 2, 2024

Apache Flink

 

https://www.youtube.com/watch?v=fYO5-6Owt0w

https://www.youtube.com/watch?v=FoypLT2W91c



https://www.youtube.com/watch?v=xu3A_7DcRgQ






  • Delivery Stream is focused on delivering data to various storage or analytics destinations with optional data transformation, ideal for near real-time ETL (Extract, Transform, Load) processes.
  • Data Stream is more about ingesting and processing real-time data at low latency, where you can build custom applications that consume and process the data in real time.

  • https://aws.amazon.com/solutions/implementations/streaming-data-solution-for-amazon-kinesis/




  • https://www.youtube.com/watch?v=-WFhxvX_XYw

    https://www.youtube.com/watch?v=K2ibvfmFh8Y





    https://www.youtube.com/watch?v=9YLOA-8UijI






    FrameworkLanguage(s) SupportedStrengthsUse Cases
    Apache Kafka StreamsJava, ScalaTight integration with Kafka, lightweightReal-time analytics, microservices, event-driven architectures
    Apache Spark StreamingJava, Scala, Python, RUnified API for batch/stream, scalabilityContinuous ETL, real-time data processing, streaming ML
    Apache StormJava, Python, Ruby, ClojureLow-latency processingReal-time analytics, complex event processing
    Apache SamzaJava, ScalaStrong state management, scalableReal-time pipelines, monitoring
    Google Cloud DataflowJava, PythonManaged service, auto-scaling, unified batch/streamCloud-native real-time analytics, ETL
    Amazon Kinesis Data StreamsJava, Python, Node.js, RubyManaged service, AWS ecosystem integrationReal-time data ingestion, log processing
    Apache NiFiJava, XMLUser-friendly, strong data lineageData flow automation, ETL
    Azure Stream AnalyticsSQL-like languageManaged service, Azure integrationIoT data processing, real-time analytics

     


    • at