Select the top Linux distribution - Check out some of the best Linux distros for daily use, programming, beginners, lightweight, etc.

Best Linux Distros that you should try in 2022

NPM is one of the great package managers of the JavaScript runtime environment
 Node.js. With the npm, you can easily install packages that are available in
 the npm registry and also maintain your dependencies in a better way.





 There can be many scenarios where you encounter a bug in any package. As a
 possible action, it is highly likely that you will report that bug in the
 Github repository of the package &amp; wait for the fix.





 Mostly the wait time is really long, you literally need to wait for the fix of
 the reported bug. This is understandable for the situations where you can
 actually wait for the fix.





 If we consider today's "agile" approach then in most cases
 you have to fix that bug as soon as possible in order to fulfil the client's
 requirements. Clearly wait is not a desirable option in these cases.





 There are many approaches where you can patch npm package. The easiest way is
 to do it by installing an npm package called "<a
 href="https://www.npmjs.com/package/patch-package"
 target="_blank"
 rel="noreferrer noopener"
 >patch-package</a
 >".




<h2>Patch npm package in simple 4 steps:</h2>



<h3>1. Installation of package</h3>


 Install the <code>patch-package</code> package via NPM or Yarn by executing
 the below commands:




NPM

<pre class="wp-block-code"><code>npm i patch-package</code></pre>

YARN

<pre
 class="wp-block-code"
><code>yarn add patch-package postinstall-postinstall</code></pre>




 You can use <code>--save-dev</code> in case of NPM or <code>--dev</code> in
 case of YARN if you don't need to run npm in production, e.g. if you're making
 a web frontend.




<h3>2. Adding postinstall script in your package.json file.</h3>


 Since we have installed <code>patch-package</code> now we need to add a
 postinstall script in the <code>package.json</code> file as below.


<pre class="wp-block-code"><code> "scripts": {
 "postinstall": "patch-package"
 }</code></pre>


 You can refer to the package documentation if you have a question in your mind
 -
 <a
 href="https://www.npmjs.com/package/patch-package#why-use-postinstall-postinstall-with-yarn"
 >Why are we using postinstall-postinstall</a
 >




<h3>3. Alter the code of the package &amp; Create a patch file.</h3>


 After executing the previous steps, You can change the code of the desired
 package inside the <code>node_modules</code> directory.





 As a next step, you need to create a patch file by executing the below
 command:




NPM:

<pre
 class="wp-block-code"
><code>npx patch-package &lt;package_name&gt; </code></pre>

YARN:

<pre
 class="wp-block-code"
><code>yarn patch-package &lt;package_name&gt; </code></pre>




 This will create a patch file inside the patches directory. For better
 tracking you can commit this patch to your github repository by executing the
 following command.


<pre class="wp-block-code"><code>git add .
git commit -m "Created a patch for bug related to Next.js Module"
git push origin &lt;remote&gt;</code></pre>



<h3>4. Final Steps</h3>


 Woho! You have created your first patch of an NPM Package. Now as the part of
 last step, you need to apply your fix to the package manager by executing the
 below command.




<pre class="wp-block-code"><code> npm install or yarn install</code></pre>



<h2>DEMO</h2>




 As part of our demo, Let's add a code in Next.js library. We will create a
 demo next project:


<pre class="wp-block-code"><code>npx create-next-app demo-next</code></pre>

<figure class="wp-block-image size-large">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Installing_Next_JS-1024x227.png"
 alt="Creating a next project"
 class="wp-image-2704"
 />
</figure>

We will open the same in an editor - visual studio code.

<figure class="wp-block-image size-large">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Visual-Studio-Code-Next-1024x589.png"
 alt="Visual Studio Code"
 class="wp-image-2705"
 />
</figure>


 As we have learned in this article, we will install the package
 <code>patch-package</code>


<figure class="wp-block-image size-full">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Patch-Package-Installation.png"
 alt="Installing patch npm package"
 class="wp-image-2706"
 />
</figure>


 We will now add the postinstall in the script section of
 <code>package.json</code> file.


<figure class="wp-block-image size-full is-resized">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Adding-PostInstall.png"
 alt="Adding postinstall in the script section of package.json"
 class="wp-image-2707"
 width="532"
 height="445"
 />
</figure>


 Let's add some code to any next package file
 <code>node_modules</code> directory.


<figure class="wp-block-image size-large is-resized">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Code-Changed-Node-Modules-1024x595.png"
 alt="Added Patch npm package"
 class="wp-image-2708"
 width="729"
 height="424"
 />
</figure>


 We will now create a patch for this change, by executing the command shown in
 the screenshot below.


<figure class="wp-block-image size-full">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Patch-Creation.png"
 alt="creating a patch"
 class="wp-image-2709"
 />
</figure>


 If you will notice your project folder structure, a new directory
 <code>patches</code> has been added.




<figure class="wp-block-image size-full">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Patch-Folder.png"
 alt="Created a Patches directory"
 class="wp-image-2710"
 />
</figure>

As a final step, lets execute the npm install for applying patch.

<figure class="wp-block-image size-full">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Npm-Install.png"
 alt="npm install"
 class="wp-image-2711"
 />
</figure>



Now we will run the application &amp; test our patch :)



<figure class="wp-block-image size-large">
 <img
 src="https://api.ittwist.com/wp-content/uploads/2021/10/Final_Demo-1024x413.png"
 alt="Testing Changes"
 class="wp-image-2712"
 />
</figure>



If you have any question, feel free to post in comment section below.


 <a href="https://ittwist.com/blog/best-linux-distros"
 >Check out our Latest Best Linux Picks</a
 >

NPM is one of the great package managers of the JavaScript runtime environment Node.js. In this article you will learn easy ways to patch npm package.

4 Easy steps to patch npm package

Flask is a micro web framework written in Python, released in April 2010, and is developed by Armin Ronacher. Heroku is a cloud platform that allows developers to deploy their apps online free of cost. This article contains step-by-step instructions to walk you through deploying your flask app to Heroku.











<h2>1. Prerequisites</h2>



The basic requirements for getting started with the deployment is:



- <a href="https://www.python.org/downloads/" target="_blank" rel="noopener noreferrer">Installing Python</a>



-<a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git" target="_blank" rel="noopener noreferrer"> Installing Git</a> (Best for Source Code &amp; Version Control Management)



- Signup for an account of Heroku:







You can create an account on Heroku by referring to the <a href="https://signup.heroku.com/" target="_blank" rel="noopener noreferrer">https://signup.heroku.com/</a> link.



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Signup_Heroku-1024x504.png" alt="Deploy flask app to Heroku - Signup on Heroku"/></figure>



<h2>&nbsp;</h2>



<h2>2. Installing Heroku CLI</h2>



You can configure almost everything from Heroku GUI but It's preferred to install Heroku CLI since it is highly flexible.



If you are using macOS, install Heroku via brew.







<code>brew tap heroku/brew &amp;&amp; brew install heroku</code>







For Linux, you can install it by executing the below command.







<code>curl https://cli-assets.heroku.com/install.sh | sh</code>







You can refer to the below guide for more information on the Installation of Heroku CLI.







<a href="https://devcenter.heroku.com/articles/heroku-cli" target="_blank" rel="noopener noreferrer">Installing Heroku CLI - macOS, Windows, and Linux</a>







<h2>3. Creating Flask application</h2>







Let's create a minimal flask application following the below steps.







<h3>Create Flask Working Directory</h3>



Create a directory of flask:







<code>mkdir &lt;app_name&gt;</code>



<code>cd &lt;app_name&gt;</code>







<h3>Creating a Virtual Environment</h3>







You can create a python virtual environment by executing the below command.







<code>python -m venv venv</code>







If you are on Linux/Mac you can execute the below command for activating your virtual environment.







<code>source /venv/bin/activate</code>







For Windows, you can execute the below command:







<code>venvScriptsactivate.bat</code>







<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Activating-Virtual-Environment.png" alt="Deploy flask app to Heroku - Activating Virtual Environment"/></figure>







<h3>Installing flask, requests, and Gunicorn</h3>







Since we have created a virtual environment, now we will proceed ahead with the installation of the flask package.



Since we also need Gunicorn (Green Unicorn) which is a Web Server Gateway Interface (WSGI) primarily used to run flask applications.







<code>pip install flask request gunicorn</code>











<div class="doyouknow" style="background:lightpink; border-radius:20px; padding:25px">
Do you know?
<div>Flask has a built-in web server but it's not suitable for production. A real web server is able to communicate with Flask application via WSGI (Web Server Gateway Interface) and Gunicorn is a common choice.</div>
</div> 



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Installing-Dependencies.png" alt="Deploy flask app to Heroku - Installing Dependencies"/></figure>



<h3>Writing Flask app</h3>



Since we have all the dependencies installed to get started writing our flask application. Let's start with creating a new file - say <code>main.py</code>.



Let's add the following contents.







<pre class="wp-block-code"><code>from flask import Flask
app = Flask(__name__)
 
@app.route('/')
def index():
 return '&lt;h1&gt;Hello World&lt;/h1&gt;'

if __name__ == "__main__":
 app.run(debug=True, port=5000)</code></pre>







<h3>Running flask application</h3>







You can locally test your application by running the following command.







<code>python main.py</code>



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Local_test.png" alt="Local Test"/></figure>







<h2>4. Preparing to Deploy flask app to Heroku</h2>



Our flask application is ready now we just need to deploy it to Heroku.







<h3>Creating requirements.txt</h3>







Now let's freeze all the requirements inside requirements.txt by executing the below command on the command line.







<code>pip freeze &gt; requirements.txt</code>







We are doing this for making sure that all of the required packages are installed while deploying the application.







<h3>Creating a Procfile for Gunicorn</h3>







For deploying the application on Heroku, we need to include a&nbsp;Procfile inside our project root directory. This will specify the command that is needed on startup for the execution by the app.



We can create a Procfile by executing the following command.







<code>web: gunicorn &lt;app_name&gt;:app</code>







Since in our case, the starting point of the application name is main.py. So we need to write- <code>web: gunicorn main:app</code>



To create the Procfile from the command line. You can simply execute the following command.



<code>echo web: gunicorn main:app &gt;&gt; Procfile</code>







<h2>5. Deploying flask application to Heroku Platform.</h2>







<h3>Login via Heroku CLI</h3>







First and foremost, we need to log in to Heroku.







<code>heroku login</code>



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Heroku_login_Command.png" alt="Heroku Login Command"/></figure>



<figure class="wp-block-image is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Login_Page_Heroku-1024x493.png" alt="Login Page Heroku" width="657" height="316"/></figure>







<h3>Creating Heroku App</h3>







You can create a Heroku app in 2 ways. One is via GUI and another one is via CLI.







<h4>Graphical Way:</h4>



From GUI you can create an application by visiting <a href="https://dashboard.heroku.com/new-app" target="_blank" rel="noopener noreferrer">https://dashboard.heroku.com/new-app</a>



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Heroku_Create_App-1024x562.png" alt="Heroku Create App"/></figure>







<h4>CLI Way:</h4>







From CLI you can create an application by executing the following command on your terminal.







<code>heroku create &lt;app-name&gt;</code>







<h3>Deploy flask app to Heroku via git</h3>



<h4>Adding Heroku App to Remote:</h4>



<pre class="wp-block-code"><code>git init
heroku git:remote -a &lt;app-name&gt;
git remote -v</code></pre>



Deploying:







<pre class="wp-block-code"><code>git add .
git commit -m "Intial Commit"
git push heroku master</code></pre>







<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Deploying_App_git.png" alt="Deploying App"/></figure>



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Deployment_Final.png" alt="Deployment Final"/></figure>



That's it for the tutorial of the "Deploy flask app to Heroku". Now your application should be live.



<a href="https://testing-app-heroku-ittwist.herokuapp.com/">https://testing-app-heroku-ittwist.herokuapp.com/</a>



<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Final_Live.png" alt="Deploy flask app to Heroku - Final Live"/></figure>



<h2>6. Deploying Heroku app via Github:</h2>



Instead of Heroku Git, you can also deploy your application to Heroku via Github.



You can push the existing Heroku application to Git Repository on Github by the following command:







<pre class="wp-block-code"><code>git add .
git commit -m "Initial Commit"
git remote add origin &lt;Remote Github Repository&gt;
git branch -M main
git push -u origin main</code></pre>







After this, you can deploy your application from the Heroku GUI, following Github as a Deployment method.&nbsp;



<figure class="wp-block-image is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/10/Heroku_Deployment_Github-1024x590.png" alt="Heroku Deployment Github" width="608" height="349"/></figure>



If you have any questions or suggestions regarding this article, please feel free to write in the comment section below. We have also some interesting articles on Linux - <a href="https://ittwist.com/blog/best-linux-distros" target="_blank" rel="noreferrer noopener">Best Linux Distros that you should try in 2021</a>.

Flask is a popular web application framework. Heroku is a cloud platform for hosting applications. In this article, learn to deploy flask app to Heroku.

Deploy flask app to Heroku

In this article, we will explore the <a href="https://spark.apache.org/" target="_blank" rel="noreferrer noopener">apache spark</a> architecture and terminologies. Before diving into the architecture let's start with the introduction &amp; basic Terminologies of Apache Spark.



 Apache spark is a beautiful framework for distributed data processing used for manipulating data at scale. Apache spark has strong support &amp; can be coded easily by using any modern programming language like Java, Scala, Python, etc. 



Spark can be easily integrated with any resource manager like YARN, MESOS, etc. Additionally, It is a highly powerful framework that makes data computations or operations at scale easier.







<h2>Interesting Facts about Apache Spark:</h2>



<ul><li>Firstly, You can easily write a code which has a potential to transform &amp; process the data at scale.</li><li>Spark is 100x faster than Hadoop MapReduce on memory &amp; 10x faster on disk.</li></ul>







<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-48-1024x563.png" alt="Spark Components" class="wp-image-2412" width="768" height="422"/></figure>







<h2>Core Concepts &amp; Terminologies:</h2>



Before beginning the architecture, let's explore the core concepts &amp; terminologies about apache spark.







1. Partitions: 



A partition in spark is a logical division of the data that is stored on a disk. It is the building block of apache spark. The collection of partitions constitutes RDD (we will explore RDD after the core concepts).



2. Job: 



A job is a code that reads the input from the HDFS or local, performs some computation on the data &amp; returns a result.







3. Stages: 



A division of Job into a unit known as a stage. Generally, the computation in spark happens in multiple stages.







4. Task:



Every stage of spark has a task associated with it. However, each task executes over one partition across an executor.







5. Execution plan:



An execution plan contains information about how execution will happen.







6. Directed Acyclic Graph or DAG:



Spark inbuilt create a DAG for optimization of the execution plan. We have covered details of DAG in the below section. 







7. Task Scheduler:



A task scheduler is having great importance in the appropriate handling of resources. Moreover, it is responsible for sending tasks to the cluster manager, executing them on the cluster, and retrying on the failure.







8. Cluster Manager:



The importance of a cluster manager is to allocate and deallocate the resources on the nodes of a cluster. Some famous cluster managers are Mesos, YARN, or Spark standalone.







9. Driver Program:



A driver program is responsible for scheduling the jobs on the cluster and communicating with the cluster manager.







10. Executors:



Executors are the actual worker nodes that execute a task as scheduled by the driver program.







11. Spark Context:



A Spark context is used for setting up a connection to the Spark cluster. A spark context is generally created in a driver program &amp; is used to create RDD, accumulator, or broadcasting variables.







<hr class="wp-block-separator"/>







We have covered all the core concepts and terminologies about Apache Spark. Now let's understand one of the important concepts of Apache Spark i.e. RDD.







<h2>RDD (Resilient Distributed Dataset):</h2>



<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Saved-1024x823.jpg" alt="RDD in Spark" class="wp-image-2425" width="768" height="617"/><figcaption> </figcaption></figure>



Resilient Distributed Dataset is the fundamental data structure of spark. I would say RDD is the heart of spark which is a fault-tolerant collection of elements that can be operated in parallel. RDD can also be cached &amp; manually partitioned. Spark keeps persistent RDD in memory in default but it can also be spill to the disk in case of low RAM.







NOTE: RDD is immutable, which means you cannot change the original one.







RDD supports 2 types of Operations:



<ol><li>Transformations</li><li>Actions</li></ol>







<h3>1. Transformations:</h3>



Transformations are the functions that take RDD as an input and return one or more RDD as an output. These are the lazy operation in RDD. Generally, it returns one or more RDD &amp; executes when an action is encountered. Correspondingly, this is one of the elegant features of spark as it improves performance.







There are two types of transformations.



<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Saved-2-1024x660.jpg" alt="Types of transformations in Spark" class="wp-image-2427" width="512" height="330"/></figure>



a. Narrow Transformations: 



In this only limited partitions are used to compute the results. In other words, An output RDD has partitions with records that originate from a single partition in the parent RDD. For Example - map, filter, etc.



Narrow transformations are always faster.



<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Saved-4-1024x869.jpg" alt="Narrow transformations in Spark" class="wp-image-2430" width="512" height="435"/></figure>



b. Wide Transformations



In wide transformation, the data required to compute the records in a single partition may reside in many partitions of the parent RDD. Hence in the case of these transformations, It might be required to shuffle data around different nodes when creating new partitions. For Example - Joins, Cartesian, etc.



Wide transformations are slower as compared to narrow transformations.



<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Saved-5-1024x938.jpg" alt="Wide Transformation in Spark" class="wp-image-2431" width="512" height="469"/></figure>



<h3>2. Actions:</h3>



As we have seen in the previous section, all of the transformations are lazy which means they do not execute right away. In contrast, actions are the functions that actually trigger the transformation execution. For Example. - count, collect, max, min, etc.







As shown in the image below, when we are applying transformations e.g. map, filter, etc, a new RDD is created. When we apply some action e.g count, reduce, max, etc. an execution occurs.



<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Unnamed-1024x703.jpg" alt="Transformations &amp; Actions - Spark" class="wp-image-2434" width="768" height="527"/></figure>



Hence, Actions are Spark RDD operations that give non-RDD values. In like manner, The values of action are stored to drivers or to the external storage system. Action&nbsp;is one of the ways of sending data from&nbsp;the Executer&nbsp;to the&nbsp;driver.&nbsp;As we have seen in the core concepts &amp; terminologies that Executors are agents that are responsible for executing a task. While the driver is a JVM process that coordinates workers and execution of the task.



<pre class="wp-block-code"><code># The below line will find out spark
import findspark
findspark.init()
# Creating Spark Context
from pyspark import SparkContext
sc = SparkContext("local", "Demonstration App")
text_file = sc.textFile("integerlist.txt")
# Below is the transformation step - Lazy Evaluation
double = text_file.map(lambda value:value*2)
# Below is the action that triggers execution
output = counts.collect()
# Stopping Spark Context
sc.stop()</code></pre>



<h2>Apache Spark Architecture &amp; Working:</h2>



We have covered the core concepts of Apache Spark, now let's understand the full architecture of Apache Spark which will give you enough insights that how Apache Spark internally works.







Apache spark uses master-slave architecture, one central coordinator, and many distributed workers. The central coordinator or master is also called the driver. The driver communicates with many slaves or workers, typically called executors.







<figure class="wp-block-image size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Unnamed-2-1024x1024.jpg" alt="Apache Spark Architecture - Ittwist" class="wp-image-2441" width="768" height="768"/></figure>







Typically, when the user submits an application using spark-submit. In spark-submit, the main() method is invoked thereby launching the driver. When the job enters the driver, it converts the code into a logical directed acyclic graph (DAG). 



Directed Acyclic Graph&nbsp;is an arrangement of&nbsp;edges and vertices. In this graph, vertices indicate RDDs, and edges refer to the operations applied on the RDD. 



When an action of RDD is called, the created DAG is submitted to DAG Scheduler, which further divides the graph into the stages of the jobs. 



In this phase, the conversion of a logical execution plan to a physical execution plan occurs. Additionally, it creates small execution units under each stage referred to as tasks. Then it collects all tasks and sends them to the cluster.







The primary responsibilities of the DAG scheduler are:



<ul><li>To complete the computation and execution of stages for a job. Additionally, Keeping track of RDDs and run jobs in minimum time and assigns jobs to the task scheduler.</li></ul>



<ul><li>Launching the tasks on executors via cluster manager.</li></ul>







The driver asks for the resources from the cluster manager. The cluster manager launches executors on behalf of the driver. The executors process the task and the result sends back to the driver through the cluster manager.







<h3>Example:</h3>



I hope you got enough clarity now. Let's revisit it all with a help of an example. Firstly, let's suppose we have only the below step in our spark program.



<pre class="wp-block-code"><code>rdd1.join(rdd2)
 .groupBy(�)
 .filter(�) </code></pre>



Now when the user will submit the application via spark-submit &amp; main() method is invoked which thereby launches a driver program. A directed acyclic graph is made where the vertices are RDD &amp; edges are the operations.



When an action is called say <code>rdd1.collect()</code>, the DAG is submitted to the DAG scheduler which thereby divides the graphs into stages. Furthermore, Small execution units are created under each stage and are submitted to the cluster where the cluster manager plays an important role for resources.



Refer to the diagram below:



<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Unnamed-4-1024x545.jpg" alt="Apache Spark Architecture - Ittwist" class="wp-image-2444"/></figure>



<blockquote class="wp-block-quote">Tip: Always remember RDD -&gt; Stages -&gt; Tasks</blockquote>



<h3>Conclusion - Apache Spark Architecture Diagram:</h3>



In conclusion, you can refer to this diagram which clearly explains all components in apache spark.



<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/Unnamed-5-1024x439.jpg" alt="Apache Spark Architecture - Ittwist" class="wp-image-2446"/></figure>







Finally, That's it for this article, if you have any queries or suggestions please feel free to write in the comment section below. 







<a href="https://ittwist.com/blog/best-linux-distros">Check out our Latest Best Linux Picks</a>

This article introduce you to Apache spark architecture and its various terminologies. Get a full insights about how Apache spark works under the hood.

Apache Spark Architecture, Working & Terminologies

<blockquote class="wp-block-quote">In this article, Let's learn how to set up GitHub personal access token. <a href="http://github.com" target="_blank" rel="noreferrer noopener">Github</a> has officially announced - Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.<cite>So you might get an error if you are using password authentication - "fatal: unable to access repo: The requested URL returned error: 403"</cite></blockquote>







<h3>Set up Github Personal Access Token:</h3>







<ol><li>Login to your Github account &amp; Go to Settings.</li></ol>



<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-39-1024x502.png" alt="Github Repo Page" class="wp-image-2387"/></figure>







2. Go to Developer Settings Tab.







<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-40-1024x502.png" alt="Github Settings" class="wp-image-2388"/></figure>







3. Click on Personal Access Token.







<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-41-1024x502.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2389"/></figure>







4. Generate a new token by clicking on the "Generate New Token" button, then you need to give a note of your token, Expiration Date &amp; Scope of the token.







<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-42-1024x502.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2390"/></figure>







5. Click on Generate token button on the bottom, then your token will be finally generated. You can copy the same.







<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-43-1024x502.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2391"/></figure>











Now let's cover how to use this personal access token for the two scenarios, one is - if you have a new repository or the other one - if you have already cloned the old repository.







<h3>Use Personal Access Token on New Repository:</h3>







If you have a brand new repository that you have just cloned, you can simply push your changes by providing your password as a token in the prompt.







<pre class="wp-block-code"><code>$ git clone https://github.com/username/repo.git
Username: enter_your_username
Password: enter_your_token</code></pre>







<figure class="wp-block-image size-full"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-44.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2393"/></figure>







<h3>Use Personal Access Token on Existing Repository:</h3>







If you have an already cloned repository you can change the URL of the same by executing the below command.







<pre class="wp-block-code"><code>git remote set-url origin https://YOUR_TOKEN@github.com/USERNAME/REPO.git</code></pre>







<figure class="wp-block-image size-full"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-45.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2397"/></figure>







Now, you can easily commit your changes, please find the below screenshot for your reference.







<figure class="wp-block-image size-full"><img src="https://api.ittwist.com/wp-content/uploads/2021/08/image-46.png" alt="Setting up GitHub Personal Access Token - Ittwist" class="wp-image-2398"/></figure>



That's it for this article, if you have any queries or suggestions please write them down in the comment section below.







<a href="https://ittwist.com/blog/best-linux-distros">Check out our Latest Best Linux Picks</a>

GitHub has officially deprecated the password authentication, this article will provide you the steps for setting up the GitHub personal access token.

How to Set up Github Personal Access Token

Anaconda is the best distribution for Python and R which is widely used by many data scientists, developers, and data engineers. In this article, we will learn how to install anaconda in Linux.

&nbsp;

<h4>Lets begin with how to install anaconda in Linux.</h4>

&nbsp;

<ol>
<li>Go to Anaconda official website using this <a href="https://www.anaconda.com/products/individual">Link</a> &amp; Download the installation files.</li>
</ol>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2140" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-73-1024x551.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2141" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-74.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

2. Execute the installer by running the below command:

&nbsp;

<pre class="wp-block-code"><code>sudo chmod 755 Package_Name
./Package_Name</code></pre>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2142" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-75.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

Then follow the instructions &amp; choose the path of your choice, default is in your <code>anaconda3</code> directory in Linux home directory.

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2143" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-76.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

After choosing directory, it will take some time for installation.

&nbsp;

Now it will ask for <code>do you wish the installer to initialize Anaconda3</code>, you can select according to your choice. If you selected no, then conda will not alter your shell scripts. In order to initialize after the installation process, you need to execute the below commands followed by a restart of the shell.

&nbsp;

<pre class="wp-block-code"><code>source your_installed_path_to_conda&gt;/bin/activate
conda init</code></pre>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2144" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-77.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2145" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-78.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2146" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-79.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

If you are using <code>zsh</code> or other shell, then you can copy the following command from <code>.bashrc</code> to <code>.zshrc</code>.

&nbsp;

Note: Make sure to change USERNAME to your username

&nbsp;

<pre class="wp-block-code"><code># &gt;&gt;&gt; conda initialize &gt;&gt;&gt;
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/USERNAME/anaconda3/bin/conda' 'shell.bash' 'hook' 2&gt; /dev/null)"
if [ $? -eq 0 ]; then
 eval "$__conda_setup"
else
 if [ -f "/home/USERNAME/anaconda3/etc/profile.d/conda.sh" ]; then
 . "/home/USERNAME/anaconda3/etc/profile.d/conda.sh"
 else
 export PATH="/home/USERNAME/anaconda3/bin:$PATH"
 fi
fi
unset __conda_setup
# &lt;&lt;&lt; conda initialize &lt;&lt;&lt;</code></pre>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2147" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-80.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

Then restart your terminal or execute the below command.

&nbsp;

<pre class="wp-block-code"><code>source .zshrc</code></pre>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2148" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-81.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

Now you can run <code>anaconda-navigator</code> to run the UI of anaconda.

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2149" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-82.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

<figure class="wp-block-image size-large"><img class="wp-image-2150" src="https://api.ittwist.com/wp-content/uploads/2021/07/image-83-1024x728.png" alt="How to Install Anaconda in Linux- Ittwist"></figure>

&nbsp;

If you are further looking at how to install tarballs in Linux you can refer to this <a href="https://ittwist.com/blog/install-softwares-in-linux-from-a-tarball/" target="_blank" rel="noreferrer noopener">article</a>. If you have any trouble in installing or have any comments, please feel free to post in the comment section below.

This article explains how to install anaconda in Linux, Ubuntu, Mint, Manjaro, or any other distribution.

How to Install Anaconda in Linux?

<figure class="wp-block-image"><img src="https://api.ittwist.com/wp-content/uploads/2019/06/ML-Curriculum-for-Self-Learners.jpg" alt="Best Machine Learning Curriculum for Beginners - Level 101" class="wp-image-1629"/></figure>



I often see many questions by people (mostly self-learners) newly coming into this grossing field that "how can we start to learn Machine Learning?", "what is the best machine learning curriculum for Beginners? "or "What is the curriculum that every self-learner must follow?".



I already created a separate post for <a href="https://ittwist.com/blog/how-to-learn-machine-learning-quickly-roadmap" target="_blank" rel="noreferrer noopener">how can I learn machine l</a><a href="https://www.ittwist.com/blog/roadmap-how-to-learn-machine-learning-quickly/">earning?</a> and this is the post for the second question i.e. what is the curriculum of machine learning for self-learners?



There are plenty of curriculums already made over the internet but I connected with various data scientists in my connections having rich experience in this field. 



So after summing up all of the discussion. I am creating this curriculum for you. You can follow that in order to master machine learning.



Note: This covers only Machine Learning, not deep learning



I have divided the curriculum into three levels.



<h3>Machine Learning Curriculum for Beginners</h3>



<ul><li>Level 1 (Complete Novice).</li><li>Level 2 (Knows Mathematics but not Data Science basics).</li><li>Level 3 (Knows Mathematics, Data Science basics but not ML).</li></ul>



According to your level of expertise, you can iterate between different levels accordingly. 



I am specifying a high-level curriculum for Level 1 and Level 2. As it is the prerequisites required for learning machine learning. 



<h2>Level 1: </h2>



The first level involves learning mathematics.



Learning Mathematics:



<ol><li>Linear Algebra</li><li>Calculus</li><li>Probability and Statistics</li></ol>







If you are wondering where to learn. Please follow:







<a href="https://ittwist.com/blog/how-to-learn-machine-learning-quickly-roadmap">How to Learn Machine Learning Quickly � A Great Roadmap</a>



<a href=""></a>



<h2>Level 2:</h2>



The second level involves learning Data Science.



<ol><li>Numpy [<a href="https://www.numpy.org/devdocs/user/quickstart.html">Link</a>]</li><li>Pandas [<a href="http://pandas.pydata.org/pandas-docs/stable/">Link</a>] [<a href="https://ittwist.com/blog/complete-pandas-tutorial-from-start-to-end-chapter-1">Chapter 1</a>] [<a href="https://ittwist.com/blog/complete-pandas-tutorial-from-start-to-end-chapter-2">Chapter 2</a>] [<a href="https://ittwist.com/blog/complete-pandas-tutorial-from-start-to-end-chapter-final">Chapter 3</a>]</li><li>Matplotlib [<a href="https://matplotlib.org/tutorials/index.html">Link</a>]</li><li>Seaborn [<a href="https://seaborn.pydata.org/tutorial.html">Link</a>]</li></ol>











<h2>Level 3</h2>



Machine Learning Curriculum



<ol><li>Importing the Dataset + Practice </li><li>Exploratory Data Analysis + Practice </li><li>Data Preprocessing + Practice </li><li>Handling Missing Data + Practice </li><li>Feature Scaling and Selection + Practice </li><li>Scikit Learn Library</li><li>Bias Variance Tradeoff</li><li>Introduction to Supervised Learning + Practice </li><li>Linear Regression with one variable + Practice </li><li>Linear Regression with multiple variables and Regularization + Practice </li><li>SVMs + Practice </li><li>Logistic Regression + Practice</li><li>Naive Bayes + Practice </li><li>Decision Tree + Practice </li><li>Introduction to Ensembles + Practice </li><li> Random Forests + Practice </li><li>K-Nearest Neighbour + Practice </li><li>PCA + Practice </li><li>Introduction Unsupervised Learning</li><li>Clustering - DBScan, KMeans + Practice </li><li> Cross-Validation and Grid Search CV + Practice </li><li>Stochastic Gradient Descent for Classification and Regression + Practice </li><li>Time Series Analysis + Practice </li><li>Bagging and Boosting Techniques + Practice </li><li>XGBoost, CATBoost, LighGBM + Practice </li><li>Kaggle Ex-Competition Practice</li></ol>







References: [<a href="https://scikit-learn.org/stable/tutorial/index.html">SciKit Learn Documentation</a>]



I will post the series of Intuitions and Jupyter notebooks in upcoming posts using the prefix "ML-Series" or you can also find them by looking into the category "Machine Learning Series". Stay Tuned for more updates.



If you have any queries or suggestions, please feel free to drop a comment below.

Machine learning is one of the grossing fields. This article Machine Learning Curriculum for Beginners, developed industry experts.

Best Machine Learning Curriculum for Beginners - Level 101

I see many people wonder how to create a symbolic link in Linux, so I thought to create an article on this topic. Symbolic links also termed soft links are very useful in Linux. A symbolic link is a special kind of file in Linux that points to another file. It's just like a shortcut if you are coming from Microsoft Windows or an alias if you are coming from Macintosh OS. In this article, we will learn how to create a symbolic link in <a href="https://linux.org/" data-type="URL" target="_blank" rel="noreferrer noopener">Linux</a>.



<h4>Lets begin with How to create a Symbolic link in Linux:</h4>



the command <code>ln</code> is used to make the link between the files. There are two types of links one is the soft link or symbolic link another one is the hard link. If we specify -s then it means that it is a soft link or symbolic link.



The usage of <code>ln</code> command is as follows:



<pre class="wp-block-code"><code>ln -s &lt;Source Directory&gt; &lt;Symbolic Link&gt;</code></pre>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-30.png" alt="How to create a Symbolic link in Linux" class="wp-image-2048"/></figure></div>



Suppose we have a directory in our home directory - say, <code>Dir</code> and we have further two directories inside that, one is A, other is B. Inside A directory there are further two directories i.e C and D. The B directory has one file inside it say, <code>abc.txt</code>. Now let's create a symbolic link between directory D and B. 



We can easily create a symbolic link with the below command.



<pre class="wp-block-code"><code>ln -s A/D B</code></pre>



A symbolic link will be created from the D directory to the B Directory.



<hr class="wp-block-separator"/>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/Symbolic-Link-Ittwist-1024x724.png" alt="How to create a Symbolic link in Linux" class="wp-image-2049" width="768" height="543"/></figure></div>



Now If you see the contents of the B Directory it has a symbolic link of D Directory.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-31.png" alt="How to create a Symbolic link in Linux" class="wp-image-2050"/></figure></div>



If you see the contents of the B Directory you will see the soft link represented by <code>D -&gt; A/D</code>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-32.png" alt="How to create a Symbolic link in Linux" class="wp-image-2051"/></figure></div>



Below is the output of the <code>ls --recursive</code> which states that due to symbolic link you have a directory D present inside the B directory but not vice-versa.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-33.png" alt="How to create a Symbolic link in Linux" class="wp-image-2052"/></figure></div>



If you delete the D directory, the parent directory will not be deleted since it is a symbolic link only.







Now, Let's take one another example of creating the files and editing it via symbolic links.



Lets create a directory inside the home directory named as <code>File</code> &amp; create a file inside that <code>hello.txt</code>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-34.png" alt="How to create a Symbolic link in Linux" class="wp-image-2053"/></figure></div>



Further let's add some content inside the <code>hello.txt</code> file.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-35.png" alt="How to create a Symbolic link in Linux" class="wp-image-2054"/></figure></div>



Time to create a symbolic link of <code>File/hello.txt</code> as <code>hi.txt</code> in home directory.



<pre class="wp-block-code"><code>ln -s File/hello.txt hi.txt</code></pre>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-36.png" alt="How to create a Symbolic link in Linux" class="wp-image-2055"/></figure></div>



Since we have created a symbolic link, now lets see the contents of <code>hi.txt</code>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-37.png" alt="How to create a Symbolic link in Linux" class="wp-image-2056"/></figure></div>



As expected, we have the same output <code>hi.txt</code> as of <code>hello.txt</code> since the symbolic link is created. Further, we will try to change the contents of <code>hi.txt</code> and see whether the contents of <code>hello.txt</code> is changed or not.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-38.png" alt="How to create a Symbolic link in Linux" class="wp-image-2057"/></figure></div>



Now lets see the contents of <code>hello.txt</code>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-39.png" alt="How to create a Symbolic link in Linux" class="wp-image-2058"/></figure></div>



Now lets try to delete the <code>hi.txt </code>file and again check if there is any impact of the same on<code> hello.txt </code>or not. 



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-40.png" alt="How to create a Symbolic link in Linux" class="wp-image-2059"/></figure></div>



There is not any impact of deleting the file, So there is one conclusion if you delete a file, only a symbolic link file will be deleted the original file will stand unchanged and if you write a text inside the symbolic link, the original file will also get changed. That's it about symbolic links.



There are lot more things we can do with symbolic links, I mostly install my own software tarballs from the web &amp; then utilize the symbolic links for creating links to directly executing them. I will cover that in the next article. If you have any queries or suggestions on this article, please write in the comment section below. 



Also if you are interested in learning how to set the Linux environment variables permanently you can visit this <a href="https://ittwist.com/blog/how-to-set-environment-variables-in-linux-permanently/" target="_blank" rel="noreferrer noopener">article</a>.

Symbolic links are useful in linux. In this article let's learn how to create a Symbolic link in Linux.

How to create a Symbolic link in Linux?

If you are coming from the Mac OS environment to Linux, then probably you have one question in mind that "Is there any good spotlight alternative for Linux?". There are many alternatives to spotlight in Linux, but <a href="https://albertlauncher.github.io/" target="_blank" rel="noreferrer noopener">Albert</a> is very ahead in this race. In this article, I will provide you the top 3 reasons why Albert is an amazing choice for you.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-92.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2166"/></figure></div>



<h4>Installation</h4>



Below are the steps, you need to execute for installing Albert. 



<ol><li>Download the Albert installation file using this <a href="https://software.opensuse.org/download.html?project=home:manuelschneid3r&amp;package=albert" target="_blank" rel="noreferrer noopener">link</a>.</li></ol>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-86-1024x560.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2160"/></figure></div>



2. Install the download File. I am using Debian so I will use deb files for Debian. If you want to learn how to install a deb files you can refer to this <a href="https://ittwist.com/blog/install-deb-files-in-linux/">article</a>. 



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-84.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2158"/></figure></div>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-85.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2159"/></figure></div>



It's Installed. You can simply execute the Albert. Now, you need to set a hotkey, you can set <code>Ctrl+Space</code> &amp; set the theme according to your choice.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-89.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2163"/></figure></div>



Let's go to the extensions tab, and give the appropriate extensions of your choice :)



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-90.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2164"/></figure></div>



Now, you can simply press <code>CTRL+Space</code> to execute Albert.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-91.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2165"/></figure></div>



<h4>Reasons why to choose Albert for Spotlight alternative for Linux</h4>



1. Nearly same experience like spotlight.



You will get nearly same experience like spotlight. You can even search web, execute terminal commands, search files, search applications &amp; use calculators.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-93.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2167"/></figure></div>



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-94.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2168"/></figure></div>



2. Multiple Themes &lt;3



<figure class="wp-block-image size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-95.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2169"/></figure>



You can choose themes of your choice whether you like Dark or Light mode, you can choose any.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-97.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2171"/></figure></div>



3. Extremely Customizable, have ability to add multiple extensions.



You can even set your own search paths for searching the files on the hard drive. One can easily customize it.



<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://api.ittwist.com/wp-content/uploads/2021/07/image-99.png" alt="Spotlight alternative for Linux - Ittwist" class="wp-image-2173"/></figure></div>



That's it! Now I think you'll be pretty convinced about using Albert. You can also check out our top picks for the <a href="https://ittwist.com/blog/best-linux-distros">best Linux distros</a>. If you have any queries, please feel free to write in the comment section below.

Looking for a spotlight alternative for Linux, we have great reasons why Albert is a great choice for you.