{"id":11037,"date":"2018-11-24T18:20:34","date_gmt":"2018-11-24T15:20:34","guid":{"rendered":"https:\/\/railsware.com\/blog\/?p=11037"},"modified":"2021-08-17T11:00:48","modified_gmt":"2021-08-17T08:00:48","slug":"python-for-machine-learning-pandas-axis-explained","status":"publish","type":"post","link":"https:\/\/railsware.com\/blog\/python-for-machine-learning-pandas-axis-explained\/","title":{"rendered":"Python for Machine Learning: Pandas Axis Explained"},"content":{"rendered":"\n

Pandas is a powerful library in a toolbox for every Machine Learning engineer. It provides two main data structures: Series<\/a> and DataFrame<\/a>.<\/p>\n\n\n\n

Many API calls of these types accept cryptical “axis” parameter. This parameter is poorly described in Pandas\u2019 documentation, though it has a key significance for using the library efficiently. The goal of the article is to fill in this gap and to provide a solid understanding of what the “axis” parameter is and how to use it in various use cases including leading-edge artificial intelligence applications<\/a>.<\/p>\n\n\n\n

\"Pandas<\/figure><\/div>\n\n\n\n

Axis in Series<\/h2>\n\n\n\n

Series<\/em> is a one-dimensional array of values. Under the hood, it uses NumPy<\/a> ndarray<\/a>. That is where the term “axis” came from. NumPy<\/em> uses it quite frequently because ndarray<\/em> can have a lot of dimensions.<\/p>\n\n\n\n

Series<\/em> object has only “axis 0” because it has only one dimension.
\"\"
The arrow on the image displays “axis 0” and its direction for the Series<\/em> object.<\/p>\n\n\n\n

Usually, in Python, one-dimensional structures are displayed as a row of values. On the contrary, here we see that Series<\/em> is displayed as a column of values.<\/p>\n\n\n\n

Each cell in Series<\/em> is accessible via index value along the “axis 0”. For our Series<\/em> object indexes are: 0, 1, 2, 3, 4. Here is an example of accessing different values:<\/p>\n\n\n\n

>>> import pandas as pd\n>>> srs = pd.Series(['red', 'green', 'blue', 'white', 'black'])\n>>> srs[0]\n'red'\n>>> srs[3]\n'white'\n<\/pre>\n\n\n\n
Axes in DataFrame<\/h2>\n\n\n\n
DataFrame<\/em> is a two-dimensional data structure akin to SQL table or Excel spreadsheet. It has columns and rows. Its columns are made of separate Series<\/em> objects. Let’s see an example:
\"\"<\/p>\n\n\n\n
A DataFrame<\/em> object has two axes: “axis 0” and “axis 1”. “axis 0” represents rows and “axis 1” represents columns. Now it’s clear that Series<\/em> and DataFrame<\/em> share the same direction for “axis 0” – it goes along rows direction.<\/p>\n\n\n\n
Our DataFrame<\/em> object has 0, 1, 2, 3, 4 indexes along the “axis 0”, and additionally, it has “axis 1” indexes which are: ‘a’<\/em> and ‘b’<\/em>.<\/p>\n\n\n\n
To access an element within DataFrame<\/em> we need to provide two indexes (one per each axis). Also, instead of bare brackets, we need to use .loc<\/em> method:<\/p>\n\n\n\n
>>> import pandas as pd\n>>> srs_a = pd.Series([1,3,6,8,9])\n>>> srs_b = pd.Series(['red', 'green', 'blue', 'white', 'black'])\n>>> df = pd.DataFrame({'a': srs_a, 'b': srs_b})\n>>> df.loc[2, 'b']\n'blue'\n>>> df.loc[3, 'a']\n8\n<\/pre>\n\n\n\n
Using \u201caxis\u201d parameter in API calls<\/h2>\n\n\n\n
There are a lot of different API calls for Series<\/a> and DataFrame<\/a> objects which accept “axis” parameter. Series<\/em> object has only one axis, so this parameter always equals  0 <\/em> for it. Thus, you can omit it, because it does not affect the result:<\/p>\n\n\n\n
>>> import pandas as pd\n>>> srs = pd.Series([1, 3, pd.np.nan, 4, pd.np.nan])\n>>> srs.dropna()\n0    1.0\n1    3.0\n3    4.0\ndtype: float64\n>>> srs.dropna(axis=0)\n0    1.0\n1    3.0\n3    4.0\ndtype: float64\n<\/pre>\n\n\n\n
On the contrary, DataFrame<\/em> has two axes, and “axis” parameter determines along which axis an operation should be performed. For example, .sum<\/em> can be applied along “axis 0”. That means, .sum<\/em> operation calculates a sum for each column:<\/p>\n\n\n\n
>>> import pandas as pd\n>>> srs_a = pd.Series([10,30,60,80,90])\n>>> srs_b = pd.Series([22, 44, 55, 77, 101])\n>>> df = pd.DataFrame({'a': srs_a, 'b': srs_b})\n>>> df\n    a    b\n0  10   22\n1  30   44\n2  60   55\n3  80   77\n4  90  101\n>>> df.sum(axis=0)\na    270\nb    299\ndtype: int64\n<\/pre>\n\n\n\n
We see, that having sum with axis=0<\/em> smashed all values along the direction of the “axis 0” and left only columns(‘a’<\/em> and ‘b’<\/em>) with appropriate sums.<\/p>\n\n\n\n
With axis=1<\/em> it produces a sum for each row:<\/p>\n\n\n\n
>>> df.sum(axis=1)\n0     32\n1     74\n2    115\n3    157\n4    191\ndtype: int64\n<\/pre>\n\n\n\n
If you prefer regular names instead of numbers, each axis has a string alias. “axis 0” has two aliases: ‘index’<\/em> and ‘rows’<\/em>. “axis 1” has only one: ‘columns’<\/em>. You can use these aliases instead of numbers:<\/p>\n\n\n\n
>>> df.sum(axis='index')\na    270\nb    299\ndtype: int64\n>>> df.sum(axis='rows')\na    270\nb    299\ndtype: int64\n>>> df.sum(axis='columns')\n0     32\n1     74\n2    115\n3    157\n4    191\ndtype: int64\n<\/pre>\n\n\n\n
Dropping NaN values<\/h3>\n\n\n\n
Let’s build a simple DataFrame<\/em> with NaN<\/em> values and observe how axis affects .dropna<\/em> method:<\/p>\n\n\n\n
>>> import pandas as pd\n>>> import numpy as np\n>>> df = pd.DataFrame({'a': [2, np.nan, 8, 3], 'b': [np.nan, 32, 15, 7], 'c': [-3, 5, 22, 19]})\n>>> df\n     a     b   c\n0  2.0   NaN  -3\n1  NaN  32.0   5\n2  8.0  15.0  22\n3  3.0   7.0  19\n>>> df.dropna(axis=0)\n     a     b   c\n2  8.0  15.0  22\n3  3.0   7.0  19\n<\/pre>\n\n\n\n
Here .dropna<\/em> filters out any row(we are moving along “axis 0”) which contains NaN<\/em> value.<\/p>\n\n\n\n
Let’s use “axis 1” direction:<\/p>\n\n\n\n
>>> df.dropna(axis=1)\n    c\n0  -3\n1   5\n2  22\n3  19\n<\/pre>\n\n\n\n
Now .dropna<\/em> collapsed “axis 1” and removed all columns with NaN<\/em> values. Columns ‘a’<\/em> and ‘b’<\/em> contained NaN<\/em> values, thus only ‘c’<\/em> column was left.<\/p>\n\n\n\n
Concatenation<\/h3>\n\n\n\n
Concatenation function with axis=0<\/em> stacks the first DataFrame<\/em> over the second:<\/p>\n\n\n\n
>>> import pandas as pd\n>>> df1 = pd.DataFrame({'a': [1,3,6,8,9], 'b': ['red', 'green', 'blue', 'white', 'black']})\n>>> df2 = pd.DataFrame({'a': [0,2,4,5,7], 'b': ['jun', 'jul', 'aug', 'sep', 'oct']})\n>>> pd.concat([df1, df2], axis=0)\n   a      b\n0  1    red\n1  3  green\n2  6   blue\n3  8  white\n4  9  black\n0  0    jun\n1  2    jul\n2  4    aug\n3  5    sep\n4  7    oct\n<\/pre>\n\n\n\n
With axis=1<\/em> both DataFrames are put along each other:<\/p>\n\n\n\n
>>> pd.concat([df1, df2], axis=1)\n   a      b  a    b\n0  1    red  0  jun\n1  3  green  2  jul\n2  6   blue  4  aug\n3  8  white  5  sep\n4  9  black  7  oct\n<\/pre>\n\n\n\n
Summary<\/h2>\n\n\n\n
Pandas<\/em> borrowed the “axis” concept from NumPy<\/em> library. The “axis” parameter does not have any influence on a Series<\/em> object because it has only one axis. On the contrary, DataFrame<\/em> API heavily relies on the parameter, because it’s a two-dimensional data structure, and many operations can be performed along different axes producing totally different results.<\/p>\n","protected":false},"excerpt":{"rendered":"
Pandas, a powerful library for Python, is a must-have tool for every machine learning developer. Check out the hands-on explanation of the Pandas \u201caxis\u201d parameter and how to use it in various cases<\/p>\n","protected":false},"author":25,"featured_media":11105,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[],"coauthors":["Sergii Boiko"],"class_list":["post-11037","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-development"],"acf":[],"aioseo_notices":[],"categories_data":[{"name":"Engineering","link":"https:\/\/railsware.com\/blog?category=development"}],"post_thumbnails":"https:\/\/railsware.com\/blog\/wp-content\/uploads\/2018\/11\/PythonMLPandas-illustration.jpg","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/11037","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/users\/25"}],"replies":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/comments?post=11037"}],"version-history":[{"count":20,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/11037\/revisions"}],"predecessor-version":[{"id":14219,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/11037\/revisions\/14219"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/media\/11105"}],"wp:attachment":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/media?parent=11037"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/categories?post=11037"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/tags?post=11037"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/coauthors?post=11037"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}