Quiz CCA175: CCA Spark and Hadoop Developer
Quiz
val rdd: RDD[Array[Byte]]
Now you have to save this RDD as a SequenceFile. And below is the code snippet.
import org.apache.hadoop.io.compress.GzipCodec
rdd.map(bytesArray => (A.get(), new
B(bytesArray))).saveAsSequenceFile('7output/path",classOt[GzipCodec])
What would be the correct replacement for A and B in above snippet.
Quiz
You have been given MySQL DB with following details.
user=retail_dba
password=cloudera
database=retail_db
table=retail_db.categories
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following activities.
1. Connect MySQL DB and check the content of the tables.
2. Copy "retaildb.categories" table to hdfs, without specifying directory name.
3. Copy "retaildb.categories" table to hdfs, in a directory name "categories_target".
4. Copy "retaildb.categories" table to hdfs, in a warehouse directory name "categories_warehouse".
1. Connect to MySQL and inspect the tables
Run:
mysql -h quickstart -P 3306 -u retail_dba -p
When prompted, enter the password cloudera. Then run:
USE retail_db;
SHOW TABLES;
SELECT * FROM categories;
SHOW TABLES lists the tables in the database, and SELECT displays the contents of categories.
2. Import categories to HDFS without specifying a destination directory
Run:
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories
Because no destination directory is specified, Sqoop uses its default HDFS location, typically /user/<hdfs-user>/categories. The HDFS username may differ from the MySQL username.
3. Import categories into a directory named categories_target
Run:
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--target-dir /user/<hdfs-user>/categories_target
The --target-dir option specifies the exact HDFS destination directory.
4. Import categories using categories_warehouse as the warehouse directory
Run:
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--warehouse-dir /user/<hdfs-user>/categories_warehouse
Unlike --target-dir, --warehouse-dir specifies a parent directory. Sqoop normally adds the table name beneath it, so the imported data will typically be in /user/<hdfs-user>/categories_warehouse/categories.
Replace <hdfs-user> with the account used to run Sqoop. The answer is identified as A in the question, but the answer choices were not included, so the commands above describe the expected solution.
Quiz
There is a parent organization called "ABC Group Inc", which has two child companies named Tech
Inc and MPTech.
Both companies employee information is given in two separate text file as below. Please do the
following activity for employee details.
Tech Inc.txt
1,Alok,Hyderabad
2,Krish,Hongkong
3,Jyoti,Mumbai
4,Atul,Banglore
5,Ishan,Gurgaon
MPTech.txt
6,John,Newyork
7,alp2004,California
8,tellme,Mumbai
9,Gagan21,Pune
10,Mukesh,Chennai
1. Which command will you use to check all the available command line options on HDFS and How
will you get the Help for individual command.
2. Create a new Empty Directory named Employee using Command line. And also create an empty
file named in it Techinc.txt
3. Load both companies Employee data in Employee directory (How to override existing file in HDFS).
4. Merge both the Employees data in a Single tile called MergedEmployee.txt, merged tiles should
have new line character at the end of each file content.
5. Upload merged file on HDFS and change the file permission on HDFS merged file,so that owner
and group member can read and write, other user can read the file.
6. Write a command to export the individual file as well as entire directory from HDFS to local file
System.
The following commands assume that the local text files are in the current local directory and that the HDFS directory will be created at /Employee. Use absolute paths for the local files if they are elsewhere.
1. Check HDFS command-line help
Display general help for the HDFS filesystem shell:
hdfs dfs -help
Get help for an individual command, such as put:
hdfs dfs -help put
Some Hadoop versions also accept the equivalent `hadoop fs` form:
hadoop fs -help
hadoop fs -help put
2. Create the HDFS directory and an empty file
Create the directory:
hdfs dfs -mkdir /Employee
Create a zero-length file named Techinc.txt inside it:
hdfs dfs -touchz /Employee/Techinc.txt
The `-touchz` command creates an empty file in HDFS. It is different from the local `touch` command.
3. Upload both companies’ employee data and overwrite existing files if necessary
Upload Tech Inc’s local file into HDFS, naming the HDFS file Techinc.txt:
hdfs dfs -put -f "Tech Inc.txt" /Employee/Techinc.txt
Upload MPTech’s file:
hdfs dfs -put -f MPTech.txt /Employee/MPTech.txt
The `-f` option forces an overwrite if the destination file already exists. This replaces the empty Techinc.txt created in step 2 with the Tech Inc employee data.
4. Merge the two local files into one file
Run this command locally:
awk '{ print }' "Tech Inc.txt" MPTech.txt > MergedEmployee.txt
This writes the contents in the requested order and prints each input line with a newline. As a result, the last line from the first file and the first line from the second file are separated by a newline, even if a source file did not end with one.
5. Upload the merged file and set its permissions
Upload the merged file to HDFS:
hdfs dfs -put -f MergedEmployee.txt /Employee/MergedEmployee.txt
Set the permissions so the owner and group can read and write, while other users can read:
hdfs dfs -chmod 664 /Employee/MergedEmployee.txt
Permission value 664 means owner: read and write; group: read and write; others: read only.
6. Export files from HDFS to the local filesystem
Export one file, such as the merged file, to the current local directory:
hdfs dfs -get /Employee/MergedEmployee.txt .
The equivalent command is:
hdfs dfs -copyToLocal /Employee/MergedEmployee.txt .
Export the entire HDFS directory to the current local directory:
hdfs dfs -get /Employee ./Employee
The final argument is the local destination. The directory will be copied locally as Employee.
Quiz
user=retail_dba
password=cloudera
database=retail_db
table=retail_db.categories
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following activities.
1. Import data from categories table, where category=22 (Data should be stored in categories
subset)
2. Import data from categories table, where category>22 (Data should be stored in
categories_subset_2)
3. Import data from categories table, where category between 1 and 22 (Data should be stored in
categories_subset_3)
4. While importing catagories data change the delimiter to '|' (Data should be stored in
categories_subset_S)
5. Importing data from catagories table and restrict the import to category_name,category id
columns only with delimiter as '|'
6. Add null values in the table using below SQL statement ALTER TABLE categories modify
category_department_id int(11); INSERT INTO categories values (eO.NULL.'TESTING');
7. Importing data from catagories table (In categories_subset_17 directory) using '|' delimiter and
categoryjd between 1 and 61 and encode null values for both string and non string columns.
8. Import entire schema retail_db in a directory categories_subset_all_tables
1. Import the row whose category_id is 22
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--where "category_id = 22" \
--target-dir categories_subset
2. Import rows whose category_id is greater than 22
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--where "category_id > 22" \
--target-dir categories_subset_2
3. Import rows whose category_id is between 1 and 22, inclusive
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--where "category_id BETWEEN 1 AND 22" \
--target-dir categories_subset_3
4. Import the categories table using a pipe delimiter
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--fields-terminated-by '|' \
--target-dir categories_subset_S
The fields-terminated-by option changes the delimiter between fields in the output files from Sqoop’s usual default to |.
5. Import only category_name and category_id, using a pipe delimiter
The scenario does not specify a target directory for this activity, so the example below uses categories_subset_5. The directory name can be changed if the question or lab specifies another one.
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--columns "category_name,category_id" \
--fields-terminated-by '|' \
--target-dir categories_subset_5
The --columns option restricts the imported data to the named columns, in the order listed.
6. Allow and insert a null value
Run the following SQL in MySQL. The insert statement in the question appears to contain typographical errors. Assuming the table’s column order is category_id, category_department_id, category_name, the corrected statements are:
ALTER TABLE categories MODIFY category_department_id INT(11);
INSERT INTO categories VALUES (0, NULL, 'TESTING');
The ALTER TABLE statement allows category_department_id to be null. The INSERT statement then adds a row with a null department ID. The value 0 for category_id is commonly used with an auto-increment column to request a generated ID, provided MySQL’s NO_AUTO_VALUE_ON_ZERO mode is not enabled.
7. Import rows with category_id between 1 and 61, use a pipe delimiter, and represent nulls
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--where "category_id BETWEEN 1 AND 61" \
--fields-terminated-by '|' \
--null-string '\\N' \
--null-non-string '\\N' \
--target-dir categories_subset_17
The null-string option specifies the representation for null string values, while null-non-string does the same for non-string values. Here, both use \N in the output.
8. Import all tables in the retail_db schema
sqoop import-all-tables \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--warehouse-dir categories_subset_all_tables
For import-all-tables, --warehouse-dir is appropriate because Sqoop creates a separate subdirectory for each imported table under the specified directory.
Each target directory must not already exist when the import runs. In a production environment, credentials would generally be supplied through a protected mechanism rather than written directly in the command.
Quiz
user=retail_dba
password=cloudera
database=retail_db
table=retail_db.categories
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following activities.
Import Single table categories(Subset data} to hive managed table , where category_id between 1
and 22
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table categories \
--where "category_id BETWEEN 1 AND 22" \
--hive-import \
--hive-database retail_db \
--hive-table categories \
--create-hive-table
The `--where` clause selects only the requested subset, including both boundary values. `--hive-import` directs Sqoop to load the imported data into Hive, and the Hive database and table options specify the destination as `retail_db.categories`. `--create-hive-table` asks Sqoop to create that table if it does not already exist.
The answer choices were not included, so I cannot verify which command is labeled A. The command above expresses the requirements described in the question.
Quiz
user=retail_dba
password=cloudera
database=retail_db
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following activities.
1. List all the tables using sqoop command from retail_db
2. Write simple sqoop eval command to check whether you have permission to read database tables
or not.
3. Import all the tables as avro files in /user/hive/warehouse/retail cca174.db
4. Import departments table as a text file in /user/cloudera/departments.
1. List the tables in retail_db:
```bash
sqoop list-tables \
--connect 'jdbc:mysql://quickstart:3306/retail_db' \
--username retail_dba \
--password cloudera
```
`list-tables` connects to the specified database and displays the tables the account can access.
2. Check whether the account can read a table:
```bash
sqoop eval \
--connect 'jdbc:mysql://quickstart:3306/retail_db' \
--username retail_dba \
--password cloudera \
--query 'SELECT * FROM departments LIMIT 1'
```
`eval` runs the SQL query against MySQL and displays its result. A successful query confirms that the account can read from `departments`; an access-denied error indicates a permissions problem. This checks access to that table, not necessarily every table in the database.
3. Import all tables as Avro data files:
```bash
sqoop import-all-tables \
--connect 'jdbc:mysql://quickstart:3306/retail_db' \
--username retail_dba \
--password cloudera \
--as-avrodatafile \
--warehouse-dir '/user/hive/warehouse/retail cca174.db'
```
`import-all-tables` imports the database tables, and `--as-avrodatafile` selects Avro as the output format. Sqoop creates a directory for each imported table beneath the warehouse directory. The quotes preserve the space in the destination path. If the intended directory name is `retail_cca174.db` with an underscore, use that spelling instead.
4. Import the departments table as a text file:
```bash
sqoop import \
--connect 'jdbc:mysql://quickstart:3306/retail_db' \
--username retail_dba \
--password cloudera \
--table departments \
--target-dir /user/cloudera/departments \
--as-textfile
```
This imports only `departments` into the specified HDFS directory as text. `--as-textfile` makes the format explicit; text is also Sqoop’s usual default import format.
Quiz
user=retail_dba
password=cloudera
database=retail_db
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Compression Codec: org.apache.hadoop.io.compress.SnappyCodec
Please accomplish following.
1. Import entire database such that it can be used as a hive tables, it must be created in default
schema.
2. Also make sure each tables file is partitioned in 3 files e.g. part-00000, part-00002, part-00003
3. Store all the Java files in a directory called java_output to evalute the further
Run:
mkdir -p java_output
sqoop import-all-tables \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--hive-import \
--hive-database default \
--num-mappers 3 \
--compress \
--compression-codec org.apache.hadoop.io.compress.SnappyCodec \
--outdir java_output
Why these options satisfy the requirements:
- `import-all-tables` imports all eligible tables from the MySQL database named in the JDBC URL.
- `--hive-import` creates the corresponding Hive tables and loads the imported data into Hive.
- `--hive-database default` places those tables in Hive’s `default` database.
- `--num-mappers 3` uses three map tasks, producing three output data files for each table under normal Sqoop/Hadoop output behavior. These files are commonly named with `part-m-` prefixes; the exact filenames can vary. This is three output files, not Hive table partitioning.
- `--compress` enables compression, and `--compression-codec org.apache.hadoop.io.compress.SnappyCodec` selects Snappy.
- `--outdir java_output` tells Sqoop to place the generated Java source files in the `java_output` directory.
This assumes the tables are eligible for Sqoop’s `import-all-tables` operation, typically meaning they have primary keys that Sqoop can use to split the imports.
Quiz
user=retail_dba
password=cloudera
database=retail_db
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following.
1. Import department tables using your custom boundary query, which import departments
between 1 to 25.
2. Also make sure each tables file is partitioned in 2 files e.g. part-00000, part-00002
3. Also make sure you have imported only two columns from table, which are
department_id,department_name
A corresponding Sqoop command is:
```bash
sqoop import \
--connect 'jdbc:mysql://quickstart:3306/retail_db' \
--username retail_dba \
--password cloudera \
--query 'SELECT department_id, department_name
FROM departments
WHERE department_id BETWEEN 1 AND 25
AND $CONDITIONS' \
--split-by department_id \
--boundary-query 'SELECT 1, 25' \
--num-mappers 2 \
--target-dir /user/cloudera/departments
```
The query names only `department_id` and `department_name`, so those are the only columns imported. The `BETWEEN 1 AND 25` condition limits the rows, and `$CONDITIONS` is required by Sqoop when using `--query` so it can add a condition for each mapper’s split.
The boundary query, `SELECT 1, 25`, supplies the lower and upper bounds used to divide the `department_id` range. `--split-by department_id` tells Sqoop which column to divide, and `--num-mappers 2` requests two parallel import tasks, normally producing two part files.
With standard Sqoop/Hadoop output naming, the files are generally named `part-m-00000` and `part-m-00001`. The exact filenames can vary by Hadoop or Sqoop version; setting two mappers controls the requested number of output partitions, not specific numeric suffixes such as `00000` and `00002`.
Quiz
Please accomplish following.
1. Import joined result of orders and order_items table join on orders.order_id =
order_items.order_item_order_id.
2. Also make sure each tables file is partitioned in 2 files e.g. part-00000, part-00002
3. Also make sure you use orderid columns for sqoop to use for boundary conditions.
A representative Sqoop command is:
sqoop import \
--connect jdbc:mysql://<host>:<port>/<database> \
--username <username> \
--password <password> \
--query "SELECT o.order_id AS order_id, o.*, oi.* \
FROM orders o \
JOIN order_items oi \
ON o.order_id = oi.order_item_order_id \
WHERE \$CONDITIONS" \
--split-by order_id \
--boundary-query "SELECT MIN(order_id), MAX(order_id) FROM orders" \
--num-mappers 2 \
--target-dir <hdfs-target-directory>
The key parts are:
- The JOIN condition matches orders.order_id with order_items.order_item_order_id.
- The $CONDITIONS placeholder is required in a Sqoop --query so Sqoop can add its mapper-specific conditions.
- --num-mappers 2 requests two parallel import tasks, which normally produce two part files. The exact filenames depend on the Hadoop output format and environment.
- --split-by order_id tells Sqoop to divide the import by the order ID.
- --boundary-query supplies the minimum and maximum order IDs Sqoop uses to determine the split ranges.
The split column should be selected by the query and should identify the same order ID used for the join. If the SQL query or Sqoop version requires fully qualified columns, use the appropriate qualified or aliased column consistently for the SELECT and --split-by settings.
Quiz
user=retail_dba
password=cloudera
database=retail_db
jdbc URL = jdbc:mysql://quickstart:3306/retail_db
Please accomplish following.
1. Import departments table in a directory.
2. Again import departments table same directory (However, directory already exist hence it should
not overrride and append the results)
3. Also make sure your results fields are terminated by '|' and lines terminated by '\n\
For example, using `/user/cloudera/departments` as the destination directory:
First import:
```bash
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table departments \
--target-dir /user/cloudera/departments \
--fields-terminated-by '|' \
--lines-terminated-by '\n'
```
Second import into the same directory:
```bash
sqoop import \
--connect jdbc:mysql://quickstart:3306/retail_db \
--username retail_dba \
--password cloudera \
--table departments \
--target-dir /user/cloudera/departments \
--fields-terminated-by '|' \
--lines-terminated-by '\n' \
--append
```
`--target-dir` specifies the HDFS directory for the imported files. The delimiter options produce pipe-separated fields and newline-terminated records. On the second command, `--append` tells Sqoop to add the newly imported data to the existing dataset rather than fail because the directory already exists or replace its contents. Since this imports the same table twice, the appended results will include another copy of the table’s rows.
CCA175: CCA Spark and Hadoop Developer Practice test unlocks all online simulator questions
Thank you for choosing the free version of the CCA175: CCA Spark and Hadoop Developer practice test! Further deepen your knowledge on Cloudera Simulator; by unlocking the full version of our CCA175: CCA Spark and Hadoop Developer Simulator you will be able to take tests with over 95 constantly updated questions and easily pass your exam. 98% of people pass the exam in the first attempt after preparing with our 95 questions.
BUY NOWWhat to expect from our CCA175: CCA Spark and Hadoop Developer practice tests and how to prepare for any exam?
The CCA175: CCA Spark and Hadoop Developer Simulator Practice Tests are part of the Cloudera Database and are the best way to prepare for any CCA175: CCA Spark and Hadoop Developer exam. The CCA175: CCA Spark and Hadoop Developer practice tests consist of 95 questions and are written by experts to help you and prepare you to pass the exam on the first attempt. The CCA175: CCA Spark and Hadoop Developer database includes questions from previous and other exams, which means you will be able to practice simulating past and future questions. Preparation with CCA175: CCA Spark and Hadoop Developer Simulator will also give you an idea of the time it will take to complete each section of the CCA175: CCA Spark and Hadoop Developer practice test . It is important to note that the CCA175: CCA Spark and Hadoop Developer Simulator does not replace the classic CCA175: CCA Spark and Hadoop Developer study guides; however, the Simulator provides valuable insights into what to expect and how much work needs to be done to prepare for the CCA175: CCA Spark and Hadoop Developer exam.
BUY NOWCCA175: CCA Spark and Hadoop Developer Practice test therefore represents an excellent tool to prepare for the actual exam together with our Cloudera practice test . Our CCA175: CCA Spark and Hadoop Developer Simulator will help you assess your level of preparation and understand your strengths and weaknesses. Below you can read all the quizzes you will find in our CCA175: CCA Spark and Hadoop Developer Simulator and how our unique CCA175: CCA Spark and Hadoop Developer Database made up of real questions:
Info quiz:
- Quiz name:CCA175: CCA Spark and Hadoop Developer
- Total number of questions:95
- Number of questions for the test:50
- Pass score:80%
You can prepare for the CCA175: CCA Spark and Hadoop Developer exams with our mobile app. It is very easy to use and even works offline in case of network failure, with all the functions you need to study and practice with our CCA175: CCA Spark and Hadoop Developer Simulator.
Use our Mobile App, available for both Android and iOS devices, with our CCA175: CCA Spark and Hadoop Developer Simulator . You can use it anywhere and always remember that our mobile app is free and available on all stores.
Our Mobile App contains all CCA175: CCA Spark and Hadoop Developer practice tests which consist of 95 questions and also provide study material to pass the final CCA175: CCA Spark and Hadoop Developer exam with guaranteed success. Our CCA175: CCA Spark and Hadoop Developer database contain hundreds of questions and Cloudera Tests related to CCA175: CCA Spark and Hadoop Developer Exam. This way you can practice anywhere you want, even offline without the internet.
BUY NOW