Get Unlimited Contributor Access to the all ExamTopics Exams!
Take advantage of PDF Files for 1000+ Exams along with community discussions and pass IT Certification Exams Easily.
The code block shown below contains an error. The code block intended to create a single-column DataFrame from Scala List years which is made up of integers. Identify the error.
Code block:
spark.createDataset(years)
A.
The years list should be wrapped in another list like List(years) to make clear that it is a column rather than a row.
B.
The data type is not specified – the second argument to createDataset should be IntegerType.
C.
There is no operation createDataset – the createDataFrame operation should be used instead.
D.
The result of the above is a Dataset rather than a DataFrame – the toDF operation must be called at the end.
E.
The column name must be specified as the second argument to createDataset.
Official Databricks tests (where answer is A)
Question 44 Which of the following code blocks creates a single-column DataFrame from Scala Listyears which is made up of integers? A. spark.createDataset(years).toDF B. spark.createDataFrame(years, IntegerType) C. spark.createDataset(years) D. spark.DataFrame(years, IntegerType) E. spark.createDataFrame(years)
C. There is no operation createDataset – the createDataFrame operation should be used instead.
The correct method to create a DataFrame in Spark using Scala is createDataFrame, not createDataset. The correct syntax would be:
scala
Copy code
val df = spark.createDataFrame(years.map(Tuple1.apply)).toDF("columnName")
This assumes that years is a List of integers, and the resulting DataFrame will have a single column named "columnName".
A voting comment increases the vote count for the chosen answer by one.
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one.
So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
deadbeef38
1 week agoSowwy1
2 months, 3 weeks agoSowwy1
2 months, 3 weeks agotangerine141
4 months, 2 weeks agozozoshanky
11 months, 1 week ago