Azure Data Factory v2 And Decompression

Kevin Feasel

2018-04-23

Bugs, Cloud, ETL

Ben Jarvis notes a file naming bug with Azure Data Factory v2 when decompressing files:

ADF V2 natively supports decompression of files as documented at https://docs.microsoft.com/en-us/azure/data-factory/supported-file-formats-and-compression-codecs#compression-support. With this functionality ADF should change the extension of the file when it is decompressed so 1234_567.csv.gz would become 1234_567.csv however, I’ve noticed that this doesn’t happen in all cases.

In our particular case the file names and extensions of the source files are all uppercase and when ADF uploads them it doesn’t alter the file extension e.g. if I upload 1234_567.CSV.GZ I get 1234_567.CSV.GZ in blob storage rather than 1234_567.CSV.

Click through for more details and be sure to vote on his Azure Feedback bug if this affects you.

Related Posts

The Value Of Power BI Dataflows

Matt Allington gets to the core benefits of Power BI Dataflows: Dataflows are: An online service provided by Microsoft as part of Power BI (software as a service, or SaaS). In effect dataflows are an online data collection and storage tool. Collection:  It uses Power Query to connect to the data at the source and transform that data as […]

Read More

Controlling Azure Services In R With AzureR

Hong Ooi announces a new set of packages called AzureR: As background, some of you may remember the AzureSMR package, which was written a few years back as an R interface to Azure. AzureSMR was very successful and gained a significant number of users, but it was never meant to be maintainable in the long term. As […]

Read More

Categories

April 2018
MTWTFSS
« Mar May »
 1
2345678
9101112131415
16171819202122
23242526272829
30