Datascience24

Using the code provided in section 4.5.1 AND 4.5.2, create an LDA model of the essays in the okc dataset. Create charts of most common terms per topic. ANSWER THE FOLLOWING QUESTIONS: 1. Explain how the html tags and newline characters were removed from the text. 2. How were the individual words combined into complete essays? 3. Explain what an LDA model does. 4. What are stop words? 5. In which 2 topics is the word 'want' not in the top ten? 6. What code snippet causes the topics...

Using the code in section 4.4.1, create a generalized linear regression model. Extract the coefficient estimates into a tidy dataframe. ANSWER THE FOLLOWING QUESTIONS: 1. What does glr represent? 2. What is the standard error of the intercept? Use the code provided to create a coefficient plot. ANSWER THE FOLLOWING QUESTIONS: 3. Which two coefficients have the tightest confidence intervals. 4. Why does the y axis appear on the bottom of the chart while the x axis is on the left side? Us...

Using the code in section 4.4 create 10 subsets of the okc_train data set. Create an analysis set and an assessment set, ANSWER THE FOLLOWING QUESTION: 1. What does sdf_random_split() do? 2. Explain the function of (rbind, vfolds[2:10]) Use the code in section 4.4 to transform the analysis set by scaling age in each of the training and validation sets by creating a function that finds mean and standard deviation. ANSWER THE FOLLOWING QUESTIONS: 3. What does the function(data) code do? Ex...

Using the code in section 4.3 to scale the age variable. ANSWER THE FOLLOWING QUESTIONS: 1. Explain what the mutate function is doing in this line of code: mutate(scaled_age = (age - !!scale_values$mean-age) / !!scale_values$sd_age) 2. What do the two exclamation marks next to each other do? Use the code in the book to create a histogram of Scaled Age ANSWER THESE QUESTION: 3. Approximately how many profiles in the training set fall in the 0 bin? Using the code in section 4.3, aggregat...

Continue to use the code in Chapter 4 to look at the relationship between predictors and the response variable. ANSWER THESE QUESTIONS: 1. Which religious group has the highest proportion of unemployed? 2. What does the 'se' column stand for? 3. What does the function regexp_extract() do? 4. Which ethnicity has the highest proportion of unemployed? Use the code provided to create a box plot of religion v. unemployed. ANSWER THE FOLLOWING QUESTIONS: 5. What does the box plot s...

Follow the instructions in the book in Chapter 4 to read in the okc data. Explain what each element in the code is doing. Glimpse() the data. ANSWER THESE QUESTIONS; 1.What is the data type of the essay fields? 2.What do they contain? 3.How do you know the data is in Spark? Continue to use the code in Chapter 4 to add a response variable. ANSWER THESE QUESTIONS: 4. What is the response variable added? 5. Explain how the response variable is aggregated. 6. What is the tally? Continue t...

The text is Mastering Spark with R. Using your sc connection and the correct Java version: 1. explain what each element is doing in the following code blocks and the difference between these two code blocks that explains the difference in the output. Explain what the output tables mean: >cars %>% + ml_linear_regression(mpg ~ .) %>% + summary() >cars %>% + ml_linear_regression(mpg ~ hp + cyl) %>% + summary() cars %>% + ml_linear_regression(mpg ~ ...

The text is Mastering Spark with R. Making sure that you are using the sc connection and the correct version of Java, install the ggplot2 library. Run the following code: >car_group <- cars %>% + group_by(cyl) %>% + summarize(mpg = sum(mpg, =true)) + collect() %>% + print() Using similar code determine the average mpg for each cylinder count. Plot both results using the following code: >ggplot(aes(r(cyl), mpg), data = car_group) + geom_col(fill = #) + coord_flip...

The text is Mastering Spark with R. Using the sc connection that you have built earlier and making sure that you are still using Java 8, install the corr library and execute the following lines of code: >ml_corr(cars) >correlate(cars, use = "", method = "pearson" >correlate(cars, use = "", method = "pearson" %>% shave() %>% rplot() Submit a Word doc with screenshots of the results of running the code. Explain what the chart is showing. Expl...

The text is Mastering Spark with R. Using the sc connection with Java 8, execute the following lines of code: >summarize(cars, mpg_percentile = percentile(mpg, 0.25) >summarize(cars, mpg_percentile = percentile(mpg, 0.25) %>% show_query() >summarize(cars, mpg_percentile = percentile(mpg, array(0.25, 0.5, 0.75) )) >summarize(cars, mpg_percentile = percentile(mpg, array(0.25, 0.5, 0.75) )) %>% mutate(mpg_percentile = explode(mpg_percentile))