User:OrenBochman/Search/Plan

Search NG Project Todo List Operational Plan Test Plan Risk Assessment NG Search Spec Search NG Analytics NLP Tools Search Tools Search Labs Configuration Lucene-search Spec Old Code Review Q&A

Milestone 1 - Working Prototyping

• Goal: Build & Deploy Protype to labs

Tasks

• Project Admin
  • Write a Test Plan
  • Finish Risks Assesment
• Development (Search engine componets are listed below)
• Prortype Deployment to Labs

Search Engine Components

• Indexing
  • reading compressed dumps
  • reading html from cache
  • integrate Html cleaning (Tika)

• User Interface
  • {{decistage|0} javascript user interface

• Admin & Deployment
  • continious integration
  • configuration management
  • puppetise environment

• Testing
  • Unit Test in CI
  • preliminary benchmarking reports via R in docs folder

Milestone 2 - Feature Compatibility with Lucene_Search 2.1

• Goal: Match most of 2.1 Features
• Use TermPositionVectors
• Fast Highlighting
• Update analysis chain to work with current api version
  • HTML Analyzer
  • Processing Wikicode - wikitokenizer
  • Lowercase
  • Hyperlink
  • Aliases
  • Title Shingles
• Language support
  • Accent Normlization
  • Snowball
  • English
  • CJK filter
  • Serbian
  • Vietnamese
  • Russian
• Wordnet
• Spelling/Did you mean
  • Admin
• JMX support
• More Benchmarking Reports
• Integrate Existing UI

Milesotne 3: Production

• Deployment to production environment

These should be operational from the prototype stage

• Shrading
• Replication
• Update Mechanism (Incremental)
  • Get Update form Metadata Repository
  • Get Data via maintenence/dump.php
  • Bittorrent based distibution of search indexes update dumps
• Configuration Managment
  • Standalone - LocalSettings.php
  • Multiple - CommonSettings.php
• realtime indexing
  • minimize edit to search time
  • update to special page on search

Phase 4: NG Features

• UI improvemnets
• More Language support
• Result Clustering support
• Result Faceting support
• Disambiguation support
• Search Analytics
• Morphological Search
• Ontology
• Semantic Search
• Entity Extraction
• Integration of NLP tools
• Memee