Skip to main content

La donnée intelligente

Search This Blog

big data: une petit infographie qui date un peu mais toujours intéressante

on December 22, 2015

Get link
Facebook
Twitter
Pinterest
Email
Other Apps

Taming Big Data

Get link
Facebook
Twitter
Pinterest
Email
Other Apps

Comments

Post a Comment

Popular Posts

How To Break DAG Lineage in Apache Spark — 3 Methods

How To Break DAG Lineage in Apache Spark — 3 Methods In this post, I will introduce you to 3 methods how you can Brak DAG Lineage in Apache Spark. It’s very possible that 1 of them you weren’t even aware of! Check if you know all 3 methods which, depending on the conditions and requirements, can save you a lot of time! What is DAG? Apache Spark Break DAG Lineage Why do we need to break DAG Lineage? Where to see the DAG graph? How do break DAG Lineage? #1: Checkpoint #2: LocalCheckpoint #3: ReCreate DataFrame / DataSet Read more: https://medium.com/@bigdataetlcom/how-to-break-dag-lineage-in-apache-spark-3-methods-6d53f9cc1dab Credits: Big Data Etls in Medium.com Photo by <a href="https://unsplash.com/@conti_photos?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Fabrizio Conti</a> on <a href="https://unsplash.com/s/photos/mountain-climbing?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Un

Scala is a maintenance nightmare ?

This post explains why Scala projects are difficult to maintain. Scala is a powerful programming language that can make certain small teams hyper-productive. Scala can also slow productivity by drowning teams in in code complexity or burning them in dependency hell. Scala is famous for crazy, complex code – everyone knows about that risk factor already. The rest of this post focuses on the maintenance burden, a less discussed Scala productivity drain. Read more: https://mungingdata.com/scala/maintenance-nightmare-upgrade/ Credit: Mr Power Photo by <a href="https://unsplash.com/@carlevarino?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Cesar Carlevarino Aragon</a> on <a href="https://unsplash.com/@carlevarino?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Simple method for choosing the number of partitions in Spark

Simple Method to choose Number of Partitions in Spark At the end of this article, you will able to analyze your Spark Job and identify whether you have the right configurations settings for your spark environment and whether you utilize all your resources. Whenever you work on a spark job, you should consider 2 things. Avoid Spill Maximize Parallelism by utilizing all the cores. Read more: https://medium.com/analytics-vidhya/simple-method-to-choose-number-of-partitions-in-spark-75315c636a94 Credit: Tharun Kumar Sekar Photo by Fran Couto

Powered by Blogger