Get Valid Databricks Generative AI Engineer Associate Exam Dumps For Quick Success
Exam : Databricks Generative AI
Engineer Associate
Title : Databricks Certified
Generative AI Engineer
Associate
https://www.passcert.com/Databricks-Gen
erative-AI-Engineer-Associate.html
1/9
, Get Valid Databricks Generative AI Engineer Associate Exam Dumps For Quick Success
1.A Generative Al Engineer has created a RAG application to look up answers to questions about a series
of fantasy novels that are being asked on the author’s web forum. The fantasy novel texts are chunked
and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with
the user’s query, and provided to an LLM for response generation. The Generative AI Engineer used their
intuition to pick the chunking strategy and associated configurations but now wants to more methodically
choose the best values.
Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and
parameters? (Choose two.)
A. Change embedding models and compare performance.
B. Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter
retrieval.
C. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the
chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the
best performance metric.
D. Pass known questions and best answers to an LLM and instruct the LLM to provide the best token
count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
E. Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most
appropriate chunk. Optimize the chunking parameters based upon the values of the metric.
Answer: C, E
Explanation:
To optimize a chunking strategy for a Retrieval-Augmented Generation (RAG) application, the Generative
AI Engineer needs a structured approach to evaluating the chunking strategy, ensuring that the chosen
configuration retrieves the most relevant information and leads to accurate and coherent LLM responses.
Here's why C and E are the correct strategies: Strategy C: Evaluation Metrics (Recall, NDCG)
Define an evaluation metric: Common evaluation metrics such as recall, precision, or NDCG (Normalized
Discounted Cumulative Gain) measure how well the retrieved chunks match the user's query and the
expected response.
Recall measures the proportion of relevant information retrieved.
NDCG is often used when you want to account for both the relevance of retrieved chunks and the ranking
or order in which they are retrieved.
Experiment with chunking strategies: Adjusting chunking strategies based on text structure (e.g., splitting
by paragraph, chapter, or a fixed number of tokens) allows the engineer to experiment with
various ways of slicing the text. Some chunks may better align with the user's query than others. Evaluate
performance: By using recall or NDCG, the engineer can methodically test various chunking strategies to
identify which one yields the highest performance. This ensures that the chunking method provides the
most relevant information when embedding and retrieving data from the vector store.
Strategy E: LLM-as-a-Judge Metric
Use the LLM as an evaluator: After retrieving chunks, the LLM can be used to evaluate the quality of
answers based on the chunks provided. This could be framed as a "judge" function, where the LLM
compares how well a given chunk answers previous user queries.
Optimize based on the LLM's judgment: By having the LLM assess previous answers and rate their
relevance and accuracy, the engineer can collect feedback on how well different chunking configurations
perform in real-world scenarios.
This metric could be a qualitative judgment on how closely the retrieved information matches the user's
2/9
Exam : Databricks Generative AI
Engineer Associate
Title : Databricks Certified
Generative AI Engineer
Associate
https://www.passcert.com/Databricks-Gen
erative-AI-Engineer-Associate.html
1/9
, Get Valid Databricks Generative AI Engineer Associate Exam Dumps For Quick Success
1.A Generative Al Engineer has created a RAG application to look up answers to questions about a series
of fantasy novels that are being asked on the author’s web forum. The fantasy novel texts are chunked
and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with
the user’s query, and provided to an LLM for response generation. The Generative AI Engineer used their
intuition to pick the chunking strategy and associated configurations but now wants to more methodically
choose the best values.
Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and
parameters? (Choose two.)
A. Change embedding models and compare performance.
B. Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter
retrieval.
C. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the
chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the
best performance metric.
D. Pass known questions and best answers to an LLM and instruct the LLM to provide the best token
count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
E. Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most
appropriate chunk. Optimize the chunking parameters based upon the values of the metric.
Answer: C, E
Explanation:
To optimize a chunking strategy for a Retrieval-Augmented Generation (RAG) application, the Generative
AI Engineer needs a structured approach to evaluating the chunking strategy, ensuring that the chosen
configuration retrieves the most relevant information and leads to accurate and coherent LLM responses.
Here's why C and E are the correct strategies: Strategy C: Evaluation Metrics (Recall, NDCG)
Define an evaluation metric: Common evaluation metrics such as recall, precision, or NDCG (Normalized
Discounted Cumulative Gain) measure how well the retrieved chunks match the user's query and the
expected response.
Recall measures the proportion of relevant information retrieved.
NDCG is often used when you want to account for both the relevance of retrieved chunks and the ranking
or order in which they are retrieved.
Experiment with chunking strategies: Adjusting chunking strategies based on text structure (e.g., splitting
by paragraph, chapter, or a fixed number of tokens) allows the engineer to experiment with
various ways of slicing the text. Some chunks may better align with the user's query than others. Evaluate
performance: By using recall or NDCG, the engineer can methodically test various chunking strategies to
identify which one yields the highest performance. This ensures that the chunking method provides the
most relevant information when embedding and retrieving data from the vector store.
Strategy E: LLM-as-a-Judge Metric
Use the LLM as an evaluator: After retrieving chunks, the LLM can be used to evaluate the quality of
answers based on the chunks provided. This could be framed as a "judge" function, where the LLM
compares how well a given chunk answers previous user queries.
Optimize based on the LLM's judgment: By having the LLM assess previous answers and rate their
relevance and accuracy, the engineer can collect feedback on how well different chunking configurations
perform in real-world scenarios.
This metric could be a qualitative judgment on how closely the retrieved information matches the user's
2/9
