If you want to check for which compressors native libraries are present in Hadoop, you can do that using checknative -a command.
$ hadoop checknative -a 18/03/16 19:47:24 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native 18/03/16 19:47:24 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library Native library checking: hadoop: true /usr/hadoop/hadoop-2.9.0/lib/native/libhadoop.so.1.0.0 zlib: true /lib/x86_64-linux-gnu/libz.so.1 snappy: true /usr/lib/x86_64-linux-gnu/libsnappy.so.1 zstd : false lz4: true revision:10301 bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1 openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)! 18/03/16 19:47:24 INFO util.ExitUtil: Exiting with status 1: ExitException
As you can see using checknative command you can check which native libraries are present in hadoop. For zlib, snappy, lz4 and bzip2 it is true, for zstd it is false. You can also see native library for Hadoop.
Related Posts
- Data Compression in Hadoop Framework
- Java Program to Compress File in gzip Format in Hadoop
- How to Read And Write Avro Files in Hadoop
- Parquet File Format in Hadoop
- What is Big Data
- Fair Scheduler in Yarn
- Frequently Used HDFS Commands
- Word Count Program Using MapReduce in Hadoop
That’s all for the topic How to Check For Which Compressors Native Libraries Are Present in Hadoop. If something is missing or you have something to share about the topic please write a comment.
You may also like
