Julia vs Python: Choosing the best with benchmarking & performance analysis

Julia vs Python, which is the best?

In this article, we’ll learn about the Julia programming language, and benchmarking it against Python and review the performance analysis.

Over the years, I have had the pleasure of working with a wide variety of programming languages. To date, some of my favorite languages include Python, Java, and C++. The majority of developers would agree with me when I say that coding in Python is extremely simple. Once you get familiar with it, you can write extremely powerful code in just a few lines. Unfortunately, it’s a bit slow during large computational tasks due to its dynamic nature and versatility. When I compared it to the other programming languages, I observed the flow control statements like the ‘for’ loop were especially drawn out. 

Recently, I heard about Julia, a high-level, high-performance, dynamic programming language. According to the news, it was inducted into the “Petaflop Club” by virtue of its Celeste application’s peak performance topping one petaflop per second. This sparked my interest, and I decided to do a little research on the language. 

What is Julia?

Julia is a free, open-source language that is extremely easy to use. This allows researchers with low technical expertise to easily create and share programs that others will be able to use for free and improve upon. Julia uses multiple dispatch as a paradigm, making it easy to express many object-oriented and functional programming patterns. It provides asynchronous I/O, debugging, logging, profiling, a package manager, and more. Julia also uses the JIT (Just in Time) – compiler which generates native machine code. The JIT compiler provides stability via multiple dispatches, which makes it easy to compile the code to an efficient one. Julia supports multiple tools, editors (Vim, Emacs, etc) as well as IDEs (Juno, Microsoft Visual Studio, etc).

While it is a general-purpose language and can be used to write any application, many of its features are well-suited for numerical analysis and computational science. Most of the articles that I read mentioned its ability to compete with Python on the basis of speed (which was comparable to that of C). I was slightly skeptical about this claim and decided to conduct a benchmark of Julia against Python’s NumPy and OpenCV. 

The benchmark I performed was based on a basic image manipulation task of adjusting the brightness of a set of images. The reason behind choosing this was to check if Julia can perform well compared to image processing libraries like OpenCV (widely used for computer vision tasks). Both NumPy and OpenCV are heavily optimized libraries that offer high-level abstraction for various computation-intensive tasks involving vectors, images/videos, etc. A total of 2 tests were run in this benchmark,

Julia against Opencv and NumPy library in python:

Here, the image manipulation task for Julia was done using built-in methods in the “Images” library and the same was done using NumPy and OpenCV in python. After running the benchmark on 4 images of different resolutions, it was observed that OpenCV and NumPy performed better than Julia. This doesn’t mean that Julia’s performance is not as good as Python, since both OpenCV and NumPy are heavily optimized libraries, it is expected that they will be faster. Also, Julia was able to perform as well as NumPy in some computational tasks like matrix addition.

Julia against Vanilla python:

Here, the image manipulation task for both Julia and Python was done using only for loops. After running the benchmark on 4 images of different resolutions, it was observed that Julia performed much better than Python. You can have a look at the benchmarking process I followed below:

Environment Setup:

Julia offers at most four active branches, namely master, unstable, stable, and LTS. For this benchmark, we used the v1.0.5 LTS release.

How to set up Julia in Linux

Download required binaries from https://julialang.org/downloads/.

wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.5-linux-x86_64.tar.gz

Extract the tarball to a preferred location

tar -C /path/to/juliadir -xvzf julia-1.0.5-linux-x86_64.tar.gz

Now add Julia to your system path. Let’s do that by adding Julia bin path to the end of ~/.bashrc or ~/.bash_profile.

nano ~/.bashrc
export PATH="$PATH:/path/to/juliadir/bin"

For the changes to take effect let’s source the file.

source ~/.bashrc

All done, you can now invoke Julia from your CLI by simply typing Julia.

Benchmark Setup:

Clone the benchmark repo

git clone https://github.com/adarsh-ops/julia_py_benchmark.git

Change into the project directory

cd julia_py_benchmark

Install necessary packages using Julia’s inbuilt package manager

Type the following in you CLI to access Julia package manager

julia

This will invoke the Julia REPL, now press ] to access the package manager. and install the required packages

add Images
add PyCall
add BenchmarkTools

The Code

The benchmark results can be obtained by running “brightness_test.jl” script in the project’s root directory. For benchmarking, Julia’s BenchmarkTools library is used. Since Julia has foreign function interfaces support for python, it was possible to embed the python script in Julia script. Julia’s PyCall library is used to interface python with Julia.

Now all the tests included in the “brightness_test.jl” script are defined in the “tests” directory.

Julia tests:

The “tests/jl_builtin.jl” script uses Julia’s broadcast operator to perform color balancing on the RGB images.

The “tests/jl_loop.jl” script uses basic for loops to perform color balancing on the RGB images.

Python tests:

The “tests/py_cv.py” script uses Python’s OpenCV library to perform the color balancing.

The “tests/py_loop.py” script uses basic for loops to perform the color balancing.

The “tests/py_np.py” script uses NumPy’s vector operations to perform the color balancing.

In all the tests, the color balancing happens within their color_balance function, except for tests/py_cv.py since we are using OpenCV’s cv2.addWeighted() function for performing the color balancing.

Run the benchmark

The benchmark involves changing the brightness of 4 images of different resolutions by a factor of 50. You can get the benchmark results by executing the brightness_test.jl file.

julia brightness_test.jl

Benchmark Results

Apart from its high performance and some good library support, what amazed me was Julia’s ability to integrate foreign function interfaces for C/Fortran, C++, Python, R, Java, and many other languages. The main challenge I faced was installing some libraries in Julia. Initially, I had some difficulties in installing the ‘Images’ library, but after trying re-installing it a couple of times, it worked. Also, Julia had an unofficial OpenCV wrapper, but I found out that it was not fully tested and updated to the latest Julia base, which is why I didn’t include it in this benchmark.

The speeds that I got during the benchmarks tests are;

Julia Loop vs Python Loop	Result
Julia Loop	739.217 ms
Python Loop	32000.340 ms

Julia vs OpenCV vs Numpy	Results
Julia	743.869 ms
Python Numpy	70.608 ms
Python Opencv	71.669 ms

	Julia built in	Julia Loop	Python Loop	Python Numpy	Python Opencv
memory estimate	423.01 MiB	423.01 MiB	336 bytes	336 bytes	336 bytes
allocs estimate	1013	1013	13	13	13
minimum time	743.869 ms (19.57% GC)	739.217 ms (19.92% GC)	32.340 s (0.00% GC)	70.608 ms (0.00% GC)	71.669 ms (0.00% GC)
median time	767.464 ms (19.33% GC)	751.942 ms (19.56% GC)	32.340 s (0.00% GC)	72.054 ms (0.00% GC)	74.788 ms (0.00% GC)
mean time	778.136 ms (21.09% GC)	754.306 ms (20.57% GC)	32.340 s (0.00% GC)	74.342 ms (0.00% GC)	75.002 ms (0.00% GC)
maximum time	822.283 ms (25.53% GC)	797.480 ms (25.84% GC)	32.340 s (0.00% GC)	92.222 ms (0.00% GC)	84.304 ms (0.00% GC)
samples	7	7	1	68	67
evals/sample	1	1	1	1	1

Julia’s performance was 10x better than that of Python and some of its numeric computations slightly outperformed NumPy, but due to the lack of good libraries like OpenCV, executing tasks related to vision, NLP, etc. were not flexible when compared to that of Python. Though Julia had some awesome libraries for image processing and NLP, they were not fully stable and feature-rich like Python’s. However, the development is still on-going and Julia may be able to beat Python in terms of library support as well in the very near future.