If you want to check for which compressors native libraries are present in Hadoop, you can do that using checknative -a command.
$ hadoop checknative -a
18/03/16 19:47:24 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
18/03/16 19:47:24 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /usr/hadoop/hadoop-2.9.0/lib/native/libhadoop.so.1.0.0
zlib: true /lib/x86_64-linux-gnu/libz.so.1
snappy: true /usr/lib/x86_64-linux-gnu/libsnappy.so.1
zstd : false
lz4: true revision:10301
bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
18/03/16 19:47:24 INFO util.ExitUtil: Exiting with status 1: ExitException
As you can see using checknative command you can check which native libraries are present in hadoop. For zlib, snappy, lz4 and bzip2 it is true, for zstd it is false. You can also see native library for Hadoop.
That’s all for the topic How to Check For Which Compressors Native Libraries Are Present in Hadoop. If something is missing or you have something to share about the topic please write a comment.
You may also like
- Data Compression in Hadoop Framework
- Fair Scheduler in Yarn
- Frequently Used HDFS Commands
- Difference Between Checked And Unchecked Exception in Java
- How to Convert File to Byte Array in Java
- Race Condition in Java