Appending Data – It is OK to be different

 

One of the more powerful features of Power BI Desktop is the query feature, which was called Power Query back when Power BI was part of Excel. Using the Query feature, if the data which you want to use is bad, has unneeded columns or contains data formatted differently than desired all of that can be readily fixed. The best thing about the query feature is that it uses the M language and records each step. Mess up a step? No problem just delete it and keep on going.

Appending in Power BI

AppendQueriesRecently I worked on a Power BI project where I needed to merge data provided in spreadsheets. The spreadsheets came from different vendors and while they contained mostly the same data, the columns were not in the same order. I wanted all of the data to reside in one table. In Query, that means that I wanted to Append the data. The files which I were merging were very wide, and I missed the fact until after I was done that some of the columns were in different order. Power BI is smart enough to figure out the order on its own. I didn’t need to change the order of the columns at all, as long as they have the same column names. Here’s an example using three different files.

 

File 1

File 1

 

Notice each of these files is a little different

 

 

 

 

 

 

 

File 2

File 2

File 3

File 3

 

 

 

 

 

 

 

 

 

 

I want to Append these files together so that all of the columns containing the like information will be in the same column. To do this the columns do not need to be reordered. As long as the column names are the same the contents will merge. I am going to need to modify File 3 to have the same file names, so I will rename Date to Expected Duration in Minutes and Location to Plant.  Since I know that File 3 came from Slingback Central, I am going to want to add that column to File 3 as well.  Othewise I will get a null value in the Maintenance ProvidAdd Custom Columner Column.  I do not need to place the column in any specific location as long as the name is the same. Renaming the columns is pretty easy.  All one needs to do is right click the column and select rename and type in the correct column name.  To add a new column, in Query select the tab Add Column and click on the Custom Column option. As you can see in the window pictured below, the text name Slingback Central has double quotes around it. If you forget to do that, you will get a syntax error.

Putting it All Together

Now that all of my queries have the same file names, I am ready to append them together.  To do that I select one of the queries and from the Home tab click on the icon on the far right side to Append Queries.  Since I want to paste three files together, I select the option for three or more files, and select all of them so that they appear on the right in the Tables to Append section of the screen.

After appending the data together, it merges all of the like columns together regardless of the order of the original files as shown below.

append

Not having to reorder columns is a great feature as it saved me a lot of time and I hope this post can do the same for someone else.

 

Yours Always

Ginger Grant

Data aficionado et SQL Raconteur

2 comments on “Appending Data – It is OK to be different

  1. Pingback: Appending Data In Power BI – Curated SQL

Leave a Reply