CS 121 QUIZ 1 STUDY GUIDE
What is information retrieval? - Answers - The activity of searching and extracting
information from a
collection of information resources.
What was the first best mechanism for advertising on the Web? - Answers - Searching
A different definition of information retrieval - Answers - Information retrieval is a field
concerned with the structure, analysis, organization, storage, searching, and retrieval of
information
Types of IR - Answers - Search, Filtering or Tracking(detecting documents of interest),
Classification(Labeling documents with pre-existing labels), Question
Answering(Beyond searching for documents, searching for answers)
Types of Search Engines - Answers - Web Search, Vertical Search, Enterprise Search,
Desktop Search, Peer-to-Peer Search
Database Engines vs. Search Engines - Answers - Database:
-Requires data to be well-structured and well-defined
-Typically involves not just text but numbers
-Requires use of formal language and logic
Search:
-Vague and varied information needs
-Informal and ad-hoc queries
-Requires engine to infer meaning of words and sentences
Retrieval Models - Answers - -Conceptual designs of what to pay attention to when
matching a query and a document
Good retrieval model: finds documents relevant to the person who submitted the query
Web IR - Answers - The Web is huge (cannot store in any centralized memory! )
•The Web changes all the time (needs to update constantly! )
•There is information to avoid ( adversarial IR! )
•One interface for hugely divergent needs
Issues with Web Search Engines - Answers - -Performance
•Response time
•Query throughput
•Indexing speed
-Speed in discovery and integration of new documents
-Coverage
-Freshness
, -Spam
Search Engineers - Answers - •Develop and maintain search engines
•Design or optimize content for search engines
Text Acquisition - Answers - •Crawler
•Feeds
•Text conversion (e.g. PDF-> text)
•Document Store
Text Transformation - Answers - •Parser = Tokenizer + Structure
•Stopping : removing words "the", "of", etc.
•Stemming : grouping similar words together (e.g. fishes->fish)
•Link extraction and analysis: how popular is a certain link
•Information Extraction (text structure)
•Classifier (topic, non-content, spam, etc.)
Index Creation - Answers - •
Corpus statistics
•
Term weighting : how important is a word in documents
•
Index inversion: doc -> term -> term -> doc
•Index distribution: essential for large indexes
User Interaction - Answers - •Query input : keywords, operators (e.g. AND), etc.
•Query transformation
-Spell checking
-Query suggestion and expansion
•Results output
-Summaries of the pages
-Highlighting important words
Ranking - Answers - •Scoring: how well each doc matches the query
•Performance optimization : decrease response time
•Distribution
-Query broker : allocate queries to different processors
Evaluation - Answers - •Logging
-Improves the search engine in the long run
•Ranking analysis
-Considering the user behavior, the search engine analyzes if the
ranking heuristics work well for its public
•Performance analysis
-Humans expect fast answers, so the search engine is monitored to
improve the performance in the long run
What is information retrieval? - Answers - The activity of searching and extracting
information from a
collection of information resources.
What was the first best mechanism for advertising on the Web? - Answers - Searching
A different definition of information retrieval - Answers - Information retrieval is a field
concerned with the structure, analysis, organization, storage, searching, and retrieval of
information
Types of IR - Answers - Search, Filtering or Tracking(detecting documents of interest),
Classification(Labeling documents with pre-existing labels), Question
Answering(Beyond searching for documents, searching for answers)
Types of Search Engines - Answers - Web Search, Vertical Search, Enterprise Search,
Desktop Search, Peer-to-Peer Search
Database Engines vs. Search Engines - Answers - Database:
-Requires data to be well-structured and well-defined
-Typically involves not just text but numbers
-Requires use of formal language and logic
Search:
-Vague and varied information needs
-Informal and ad-hoc queries
-Requires engine to infer meaning of words and sentences
Retrieval Models - Answers - -Conceptual designs of what to pay attention to when
matching a query and a document
Good retrieval model: finds documents relevant to the person who submitted the query
Web IR - Answers - The Web is huge (cannot store in any centralized memory! )
•The Web changes all the time (needs to update constantly! )
•There is information to avoid ( adversarial IR! )
•One interface for hugely divergent needs
Issues with Web Search Engines - Answers - -Performance
•Response time
•Query throughput
•Indexing speed
-Speed in discovery and integration of new documents
-Coverage
-Freshness
, -Spam
Search Engineers - Answers - •Develop and maintain search engines
•Design or optimize content for search engines
Text Acquisition - Answers - •Crawler
•Feeds
•Text conversion (e.g. PDF-> text)
•Document Store
Text Transformation - Answers - •Parser = Tokenizer + Structure
•Stopping : removing words "the", "of", etc.
•Stemming : grouping similar words together (e.g. fishes->fish)
•Link extraction and analysis: how popular is a certain link
•Information Extraction (text structure)
•Classifier (topic, non-content, spam, etc.)
Index Creation - Answers - •
Corpus statistics
•
Term weighting : how important is a word in documents
•
Index inversion: doc -> term -> term -> doc
•Index distribution: essential for large indexes
User Interaction - Answers - •Query input : keywords, operators (e.g. AND), etc.
•Query transformation
-Spell checking
-Query suggestion and expansion
•Results output
-Summaries of the pages
-Highlighting important words
Ranking - Answers - •Scoring: how well each doc matches the query
•Performance optimization : decrease response time
•Distribution
-Query broker : allocate queries to different processors
Evaluation - Answers - •Logging
-Improves the search engine in the long run
•Ranking analysis
-Considering the user behavior, the search engine analyzes if the
ranking heuristics work well for its public
•Performance analysis
-Humans expect fast answers, so the search engine is monitored to
improve the performance in the long run
