Can two lists be added?

Combining strings or characters in programming is called concatenation. In Python, concatenation can be performed on numbers, strings and elements of a list. For example, you can add a string “Hey” and another string “there!” to form a new string “Hey there!”. You can also add two numbers such as 2 and 5 to get a result 7. The most common form of concatenation is by using the + operator.

Concatenating or merging two or multiple lists is a common operation of a programmer. In this tutorial, we will concatenate lists with and without duplicate elements.

1) Using + Operator 

Its a very common operator to concatenate lists it simply adds list behind another list.

Example:

Output:

Note:

• It retains the order of list elements
• Contain duplicate elements
• Concatenated two or more lists

Explanation

Here, the three lists are combined together using + operator. This is used because the + operator is one of the easiest ways to combine multiple lists into a single one.   

2) Using extend() Function 

extend() function is also used to concatenate lists, it simply adds the whole list at the end of the first list.

Example:

Output:

Note: 

• It retains the order of list elements
• Contain duplicate elements
• It only concatenates two lists; if you want to concatenate more then two lists, then you need to use extend() function multiple times, as shown in the above example. 

In this program, the extend() method is used for concatenating all three lists together. You can see that the extend() method is used two times for merging the second and third list with the first list. As mentioned earlier, the method cannot add more than two lists. Thus it is used two times for adding the first list with the second and third list.          

3) Using (*) Operator

(*) Operator works the same as (+) operator, with this we can concatenate to or more list it works with Python 3.6+ versions.

Example:

Output:

Note:

• It retains the order of list elements
• Contain duplicate elements
• Concatenated two or more lists

Explanation

Here the (*) operator is used for merging the three lists together while maintaining the order of the elements. The (*) operator is utilized here as multiple lists can be easily added using it.     

4) Using itertools.chain()

Example:

Output

Note: 

• Works with numpy array
• It retains the order of list element
• Contain duplicate elements
• Concatenated tow or more lists

Explanation

In the first line, the import keyword is used to load the itertools module. The variables list1, list2 and list3 are assigned values [2,3,4,2,2], [4,5,6,7,34,56] and [1,5,33,2,34,46] respectively. Then, a newlist variable is assigned the concatenated values of the three lists.

In the next line, the itertools.chain() method is passed the arguments of list1, list2 and list3. This method takes multiple arguments and returns a single sequence of items. So, the chain() method concatenates all three lists. The result of the method call is converted into a list using the list() method.

The newlist variable is printed in the last line of the program using the code print('Concatenated List: ',newlist). 

To concatenate two lists, we will first traverse the second list using a for loop. We will keep appending the elements of this list to the first list. As a result, the two lists will be added.  

Example:

Output:

Explanation

Let us try to understand this code. Here, a for loop is used for adding the two lists using the append() method. This process is used as the append() method adds a single element to the end of a list. Thus, each element of the list2 is added to list1, one by one using the for loop.

Single Line Code Example:

Output:

Explanation

We can observe that the shorthand version of a for loop is used here for concatenation. This list comprehension technique is used to avoid writing multiple inner loops, as there are three lists that have to be added. As a result, the process is fast and efficient.

Merge or Concatenate Lists Without Duplicates

From all the above examples, we can see that the final concatenate list has duplicate elements, if we want to remove duplicate from the list then we need to convert the list to “set” and then convert back to “list.”

Example:

Output:

Explanation

You can see that the main aim of the program is used to merge three lists that do not have duplicate elements. So, after concatenating the lists using a + operator, the resultant list is passed to the in-built set() method. As Python sets do not have any duplicate elements, this removes all the duplicates from the concatenated list. As we require a list, this set is converted into a list using the list() method.     

Conclusion

Concatenation is one of the easiest ways to combine list elements or strings while codding in Python. But while using the chains() method, import the itertools module using the import statement. Additionally, you can also use methods such as join() to concatenate two strings.

Problem: You have two lists and you’d like to join them into a new list. Solution:

Python 3.8.2 >>> one = ["one","two", "three"] >>> two = ["four","five"] >>> one + two ['one', 'two', 'three', 'four', 'five']

The edge cases below are better in some situations, but + is generally the best choice. All options covered work in Python 2.3, Python 2.7, and all versions of Python 31.

Combine Lists In Place In Python

Problem: You have a huge list, and you want to add a smaller list on the end while minimizing memory usage.

In this case, it may be best to append to the existing list, reusing it instead of recreating a new list.

>>>longlist = ["one","two", "three"] * 1000 ['one', 'two', 'three', 'one', 'two', 'three', ... ] >>>shortlist = ["four","five"] ["four","five"] >>> x.extend(y) >>> x ['one', 'two', 'three', 'one', ..., "four","five"]

As with any optimization, you should verify that this reduces memory thrash in your specific case and stick to the simple idiomatic x + y otherwise.

Let’s use the timeit module to check some performance numbers.

# Performance Check >>> setup = """\ x = ["one","two","three"] * 1000 y = ["four","five","six"] """ # x + y with large x >>> timeit.timeit('x + y', setup=setup, number=1000000) 3.6260274310000113 # x.extend(y) with large x >>> timeit.timeit('x.extend(y)', setup=setup, number=1000000) 0.06857255800002804

In this example, where x is 3000 elements, extend is around 50x faster.

It is possible to use chain from itertools to create an iterable of two lists.

>>>longlist = ["one","two", "three"] * 1000 ['one', 'two', 'three', 'one', 'two', 'three',, .......... ] >>>shortlist = ["four","five"] ["four","five"] >>> from itertools import chain >>> z = list(chain(longlist, shortlist) ['one', 'two', 'three', 'one', , .........., "four","five"]

We can check the performance of using chain:

>>> setup = """\ from itertools import chain x = ["one","two","three"] * 1000 y = ["four","five","six"] """ # x + y with large x # x.extend(y) with large x >>> timeit.timeit('x.extend(y)', setup=setup, number=1000000) 0.06857255800002804 >>> timeit.timeit('list(chain(x, y))', setup=setup, number=1000000) 16.810488051999982

Using chain with two lists is slower in all cases tested, and x + y is easier to understand.

Combining N Lists in Python

If you need to add three or even ten lists together and the lists are statically known, then + for concatenate works great.

>>> one = ["one","two", "three"] >>> two = ["four","five"] >>> three = [] >>> z = one + two + three

Flatten a List of Lists in Python

However, if the number of lists is dynamic and unknown until runtime, chain from itertools becomes a great option. Chain takes a list of lists and flattens it into a single list.

>>> l = [["one","two", "three"],["four","five"],[]] * 99 [['one', 'two', 'three'], ['four', 'five'], [], ... >>> list(chain.from_iterable(l)) ['one', 'two', 'three', 'four', 'five', 'one', 'two', ... ]

chain can take anything iterable, making it an excellent choice for combining lists, dictionaries, and other iterable structures.

>>> from itertools import chain >>> one = [1,2,3] >>> two = {1,2,3} >>> list(chain(one, two, one)) [1, 2, 3, 1, 2, 3, 1, 2, 3]

Performance of Flattening a List of Lists

Performance doesn’t always matter, but readability always does, and the chain method is a straightforward way to combine lists of lists. That said, let’s put readability aside for a moment and try to find the fastest way to flatten lists.

One option is iterating ourselves:

result = [] for nestedlist in l: result.extend(nestedlist)

Let’s check its performance vs chain:

>>> setup = """\ from itertools import chain l = [["one","two", "three"],["four","five"],[]] * 99 """ >>> # Add Nested Lists using chain.from_iterable >>> timeit.timeit('list(chain.from_iterable(l))', setup=setup, number=100000) 1.0384087909997106 >>> ### Add using our own iteration >>> run = """\ result = [] for nestedlist in l: result.extend(nestedlist) """ >>> timeit.timeit(run, setup=setup, number=100000) 1.8619721710001613

This shows that chain.from_iterable is faster than extend.

Flattening and Merging Lists With One Big List

What about adding a list of lists to an existing and large list? We saw that using extend can be faster with two lists when one is significantly longer than the other so let’s test the performance of extend with N lists.

First, we use our standard chain.from_iterable.

>>> # Method 1 - chain.from_iterable >>> longlist = ["one","two", "three"] * 1000 >>> nestedlist = [longlist, ["one","two", "three"],["four","five"],[]] >>> list(chain.from_iterable(nestedlist))

We then test its performance:

>>> setup = """\ from itertools import chain longlist = ["one","two", "three"] * 1000; combinedlist = [longlist, ["one","two", "three"],["four","five"],[]] """ >>> timeit.timeit('list(chain.from_iterable(combinedlist))', setup=setup, number=100000) 1.8676087710009597

Next, let’s try concatenating by adding everything onto the long list:

>>> # Method 2 - extend >>> longlist = ["one","two", "three"] * 1000 >>> nestedlist = [["one","two", "three"],["four","five"],[]] >>> for item in nestedlist: >>> longlist.extent(item)

Performance Test:

>>> setup = """\ from itertools import chain longlist = ["one","two", "three"] * 1000; nestedlist = [["one","two", "three"],["four","five"],[]] """ >>> run = """\ for item in nestedlist: longlist.extend(item) """ >>> timeit.timeit(run, setup=setup, number=100000) 0.02403609199973289

There we go, extend is much faster when flattening lists or concatenating many lists with one long list. If you encounter this, using extend to add the smaller lists to the long list can decrease the work that has to be done and increase performance.

These are the main variants of combining lists in python. Use this table to guide you in the future.

Also, if you are looking for a nice way to standardize the processes around your python projects – running tests, installing dependencies, and linting code – take a look at Earthly for Repeatable Builds.