Skip to main content
Glama
AI_RESEARCH_WORKFLOW.mdβ€’15.7 kB
# AI-Powered Research Workflow for Development **Architect First: Systematic Information Gathering Before Implementation** [![Research](https://img.shields.io/badge/AI-Research-blue)](README.md) [![Methodology](https://img.shields.io/badge/Development-Methodology-green)](README.md) [![Web Scraping](https://img.shields.io/badge/Web-Scraping-orange)](README.md) --- ## 🎯 **The "Architect First" Principle** **Before writing a single line of code, let AI be your research architect.** Build comprehensive domain knowledge through systematic information gathering. Implementation becomes dramatically easier with a solid research foundation. ### **Why Research First?** - **Avoid dead ends** - Know which approaches actually work - **Understand constraints** - Technical limitations become clear upfront - **Find communities** - Connect with experienced developers - **Discover alternatives** - Multiple solution paths emerge - **Save time** - Don't reinvent solutions that already exist - **Reduce risk** - Make informed technical decisions ### **Vibegineering Philosophy (Simon Willison)** **"Vibegineering" is the professional, responsible use of AI coding assistants** - the grown-up version of "vibe coding". Instead of accepting whatever AI spits out, vibegineering means: - **Engineer first, code second** - Set up architecture, testing, constraints - **AI handles typing, humans handle thinking** - AI writes code, experts make decisions - **Quality through process** - Professional engineering practices with AI acceleration - **Research + scaffolding = success** - This workflow embodies vibegineering principles *Reference: [JustHTML is a fascinating example of vibe engineering in action](https://simonwillison.net/2025/Dec/14/justhtml/)* ### **Traditional vs. AI-Research Workflow:** #### **Vibecoding (Naive AI Usage):** ``` 1. Have idea based on vibes/intuition 2. Ask AI to "just write the code" 3. Accept whatever AI spits out (no review) 4. Hit bugs and technical debt 5. Debug frantically without understanding 6. Eventually stumble into working solution 7. High stress, wasted time, fragile code ``` #### **Vibegineering (Simon Willison's Positive Approach):** ``` 1. Have idea with professional intent 2. Engineer/research/scaffold first (comprehensive) 3. Set up proper testing, architecture, constraints 4. Use AI as coding assistant (not replacement) 5. Expert review and quality assurance 6. Systematic development with solid foundations 7. Reliable, maintainable, high-quality results ``` #### **AI-Research Approach (Antidote to Vibegineering):** ``` 1. Have idea 2. AI research phase (comprehensive, systematic) 3. Clear technical path identified upfront 4. Implementation follows proven best practices 5. Smooth development with predictable outcomes 6. Higher success rate, lower stress, future-proof ``` --- ## 🧠 **AI Research Workflow (4-Phase Process)** ### **Phase 1: Domain Mapping** **Understand the ecosystem and landscape** ``` Prompt AI: "Map the [DOMAIN] ecosystem for [APPLICATION]" - Standards organizations and specifications - Official repositories and documentation - Current limitations and workarounds - Industry best practices and patterns - Success stories and case studies - Regulatory considerations and compliance ``` **Example:** `"Map the robotics middleware ecosystem for Unity integration"` ### **Phase 2: Tool & Technology Discovery** **Find all relevant tools, libraries, and frameworks** ``` Prompt AI: "Find all [TECHNOLOGY] tools and libraries for [PLATFORM]" - Official SDKs and APIs - Community libraries and frameworks - Alternative implementations - Integration tools and bridges - Development environments and IDEs - Testing and debugging tools ``` **Example:** `"Find all computer vision libraries for real-time robot navigation"` ### **Phase 3: Community Intelligence** **Research communities, forums, and support networks** ``` Prompt AI: "Research [DOMAIN] communities and resources" - GitHub repositories with examples - Stack Overflow and technical forums - Subreddits and specialized communities - Discord servers and Slack channels - YouTube channels and tutorial creators - Academic papers and research groups - Industry blogs and newsletters ``` **Example:** `"Research computer vision communities for robotics applications"` ### **Phase 4: Web Scraping Integration** **Systematically collect comprehensive information** ``` Prompt AI: "Use web scraping to collect comprehensive [DOMAIN] information" - Scrape official documentation sites - Collect community forum discussions - Gather tutorial links and resources - Extract code examples and snippets - Build knowledge base of best practices - Monitor for latest developments ``` **Example:** `"Use web scraping to collect ROS2 Unity integration examples"` --- ## πŸ•·οΈ **Web Scraping as Research Tool** ### **Tool Selection Strategy:** #### **BrightData MCP Tools (Premium):** - **Best for:** Restricted sites, CAPTCHAs, EU popups, geo-blocks - **Use when:** Need to access locked documentation or forums - **Cost:** Freemium ($5 free credits, then pay-as-you-go) - **Strengths:** Bypasses all entry barriers automatically #### **Fetch MCP Tool (Free):** - **Best for:** Open websites, GitHub, documentation sites - **Use when:** Sites are accessible without restrictions - **Cost:** Completely free - **Strengths:** Simple, reliable for public content #### **Manual Browsing (Fallback):** - **Best for:** When automated tools fail - **Use when:** Need human judgment or interactive elements - **Cost:** Time investment - **Strengths:** Can handle complex authentication ### **Scraping Strategy by Content Type:** | Content Type | Primary Tool | Secondary Tool | Notes | |-------------|--------------|----------------|-------| | **GitHub repos** | Fetch | Manual | Most repos are accessible | | **Official docs** | Fetch | BrightData | Some docs have restrictions | | **Forums/Reddit** | BrightData | Manual | Often have anti-bot measures | | **Research papers** | Fetch | BrightData | Some require subscriptions | | **Tutorials** | Fetch | Manual | Usually open access | | **APIs/SDKs** | Fetch | BrightData | Documentation may be restricted | --- ## πŸ“š **Research Output Structure** ### **Knowledge Base Organization:** ``` Project_Research/ β”œβ”€β”€ domain_overview.md # Ecosystem mapping β”œβ”€β”€ technical_specs.md # Standards and specifications β”œβ”€β”€ tools_and_libraries.md # Development tools β”œβ”€β”€ community_resources.md # Forums, Discord, Reddit β”œβ”€β”€ tutorials_and_guides.md # Learning materials β”œβ”€β”€ code_examples.md # Implementation snippets β”œβ”€β”€ success_stories.md # Case studies and examples β”œβ”€β”€ limitations_and_gotchas.md # Known issues and workarounds β”œβ”€β”€ cost_analysis.md # Budget considerations └── future_roadmap.md # What's coming next ``` ### **Tool Evaluation Matrix:** **For each tool/library, evaluate:** - **Compatibility** with your requirements - **Community support** and activity level - **Documentation quality** and completeness - **Learning curve** and ease of adoption - **Cost and licensing** considerations - **Integration complexity** with existing stack - **Maintenance status** and update frequency - **Security and reliability** track record --- ## 🎯 **Application to Different Domains** ### **Robotics Development:** ``` Phase 1: Map robotics middleware (ROS, ROS2, middleware alternatives) Phase 2: Find Unity integration tools, simulation platforms Phase 3: Research robotics communities, forums, Discord servers Phase 4: Scrape ROS documentation, Unity robotics tutorials ``` ### **Computer Vision:** ``` Phase 1: Map CV libraries (OpenCV, TensorFlow, PyTorch, specialized) Phase 2: Find real-time processing tools, camera integrations Phase 3: Research CV communities, research papers, conferences Phase 4: Scrape model zoos, implementation guides, performance benchmarks ``` ### **Web Development:** ``` Phase 1: Map framework ecosystem (React, Vue, Angular, backend options) Phase 2: Find deployment platforms, database solutions, APIs Phase 3: Research dev communities, Stack Overflow patterns Phase 4: Scrape documentation, tutorial sites, code examples ``` ### **Game Development:** ``` Phase 1: Map game engines (Unity, Unreal, Godot, custom) Phase 2: Find asset stores, middleware, development tools Phase 3: Research game dev communities, indie forums Phase 4: Scrape engine documentation, asset marketplaces ``` --- ## πŸ“Š **Success Metrics** ### **Research Quality Indicators:** - **Comprehensive coverage** - No major knowledge gaps - **Multiple perspectives** - Different approaches considered - **Current information** - Recent developments included - **Practical examples** - Real code and implementations - **Community validation** - Active user bases confirmed ### **Implementation Readiness:** - **Clear technical path** identified - **Risks and challenges** understood upfront - **Resource requirements** estimated - **Timeline realistic** based on research - **Backup plans** available for contingencies ### **Time Investment Return:** - **Development speed** - Faster implementation - **Fewer rewrites** - Better initial architecture - **Higher success rate** - Informed decision making - **Long-term maintainability** - Future-proof choices --- ## πŸš€ **Integration with Development Workflow** ### **Pre-Development Phase:** ``` 1. Idea conception 2. AI research phase (comprehensive) 3. Technical specification writing 4. Architecture design 5. Technology stack selection 6. Development planning ``` ### **During Development:** ``` - Reference research knowledge base - Validate assumptions against research findings - Course-correct based on discovered information - Expand research as new questions arise ``` ### **Post-Development:** ``` - Update knowledge base with lessons learned - Contribute back to communities discovered - Document solutions for future reference - Share research methodology with team ``` --- ## πŸ› οΈ **Tools and Automation** ### **AI Research Assistants:** - **ChatGPT/Claude** - General research and synthesis - **Perplexity** - Real-time web search and summarization - **GitHub Copilot** - Code example research - **Specialized AI** - Domain-specific research assistants ### **Web Scraping Automation:** - **BrightData MCP** - Premium scraping with barrier bypass - **Fetch MCP** - Free scraping for accessible sites - **BeautifulSoup/Scrapy** - Custom scraping scripts - **Browser automation** - Puppeteer/Playwright for dynamic content ### **Knowledge Management:** - **Notion/Obsidian** - Research note organization - **GitHub Wikis** - Project documentation - **Jupyter Notebooks** - Research analysis and prototyping - **Draw.io/Miro** - Architecture diagramming --- ## πŸ“ˆ **ROI and Benefits** ### **Time Savings:** - **Avoid wrong technology choices** - Research prevents 2-3 month rewrites - **Faster ramp-up** - Know exactly what to learn - **Reduced debugging** - Understand common pitfalls upfront - **Efficient problem solving** - Have multiple solution paths ready ### **Quality Improvements:** - **Better architecture** - Informed design decisions - **Fewer bugs** - Understand limitations and edge cases - **Higher reliability** - Choose proven, well-supported tools - **Future-proofing** - Consider roadmap and long-term viability ### **Risk Reduction:** - **Technical debt awareness** - Know maintenance implications - **Scalability planning** - Research performance characteristics - **Security considerations** - Understand security implications - **Compliance requirements** - Research regulatory landscape --- ## 🎯 **When to Apply This Methodology** ### **Always Use For:** - **New technology domains** you're unfamiliar with - **Complex integrations** between different systems - **Emerging technologies** with rapidly changing landscapes - **High-stakes projects** where failure is expensive - **Team projects** where knowledge sharing is important ### **Optional For:** - **Well-known domains** you have deep experience in - **Simple projects** with clear requirements - **Proof-of-concept** work where speed matters more than perfection - **Maintenance tasks** on existing, well-understood systems ### **Skip For:** - **Trivial tasks** that require no research - **Emergency fixes** where time is critical - **Pure coding exercises** or learning projects --- ## πŸ“š **Examples and Case Studies** ### **Case Study: Birthday Reminder Tool (Vibe to Vibegineering)** ``` Vibe: "I always forget friends and family birthdays, I need a tool to remind me, calendar in Windows ain't good enough" Phase 1: Domain Mapping - Research existing birthday reminder solutions (apps, services, integrations) - Study calendar APIs (Google Calendar, Outlook, Apple Calendar) - Analyze notification systems (Windows notifications, email, SMS, smart home) Phase 2: Tool & Technology Discovery - Calendar APIs: Google Calendar API, Microsoft Graph, Apple Calendar - Notification options: Windows Toast, email services, Twilio SMS, IFTTT - Data sources: social media APIs, contact imports, CSV uploads - Storage: local SQLite, cloud database, or simple JSON files Phase 3: Community Intelligence - Birthday reminder subreddits and forums - Stack Overflow questions about calendar integrations - GitHub repositories for birthday reminder apps - Reviews of existing birthday reminder services Phase 4: Web Scraping Integration - Scrape documentation for calendar APIs - Collect examples of notification implementations - Gather user feedback on existing solutions - Extract code snippets for calendar integrations Result: Comprehensive birthday reminder system with multiple notification channels, social media integration, and automated contact importing. Not just "good enough" - actually solves the core problem with redundancy and reliability. Implementation: Morning alarm with "whooop-whooop-whooop" audio signal, followed by spoken message "Georgina's birthday today! Don't forget to congratulate", ending with "whooop-whooop-whooop". Multiple notification layers ensure the reminder gets through. ``` ### **Case Study: Robotics Middleware Selection** ``` Problem: Choosing between ROS, ROS2, and alternatives for Unity robotics Research: Mapped entire robotics middleware ecosystem Result: Clear understanding of ROS2 + Unity integration path Benefit: Saved 3 months of wrong technology choices ``` ### **Case Study: Computer Vision Pipeline** ``` Problem: Real-time object detection for robot navigation Research: Evaluated OpenCV, TensorFlow, PyTorch, specialized libraries Result: Optimal pipeline with performance benchmarks Benefit: 40% faster development, better performance ``` ### **Case Study: Web Platform Architecture** ``` Problem: Choosing tech stack for complex web application Research: Mapped frontend/backend ecosystems, deployment options Result: Future-proof architecture with clear migration paths Benefit: Scalable foundation for 5-year roadmap ``` ### **Case Study: Vienna Makerspace Discovery** ``` Vibe: "I want to prototype robot parts but don't have access to fabrication equipment" Research: AI searched for Vienna makerspaces and fabrication facilities Discovery: Vienna Technical Museum's techLAB - free professional equipment! Result: Complete makerspace guide with equipment list, costs, booking system Benefit: Access to industrial 3D printers, laser cutters, CNC, electronics - for free! ``` --- **This AI-powered research methodology transforms development from trial-and-error coding to informed, strategic implementation!** πŸ§ πŸš€

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/sandraschi/robotics-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server