The Ultimate List of AI Product Design and Development Resources

This list collects and organizes a full-chain of resources for AI product design and development, targeting AI developers and product designers. It introduces the features of each resource category and provides domain addresses for specific tools. You are welcome to use and share them. You can click on the resource index at the top of the article to quickly navigate to the corresponding resource descriptions.

---

AI Resource Index

I. Model & Algorithm Core

This category of resources focuses on the "engine" of AI—the model itself.

• 1.1 Model Repositories/Platforms

• 1.2 Model Training & Fine-tuning

• 1.3 Prompt Engineering & Optimization

• 1.4 Model Evaluation & Benchmarking

II. Software Development & Engineering

This category of resources focuses on how to transform models into usable services and applications.

• 2.1 Development Frameworks & SDKs

• 2.2 Interfaces & Communication

• 2.3 Data Management & Processing

• 2.4 Deployment & Operations

III. Tools & Platforms

This category of resources focuses on ready-to-use, integrable AI services and products.

• 3.1 Model-as-a-Service (MaaS)

• 3.2 AI-Enabled Tools

• 3.3 Evaluation & Market Analysis Platforms

IV. Design & Human-Computer Interaction

This category of resources focuses on making AI applications usable, trustworthy, and efficient.

• 4.1 AI Interaction Design Patterns

• 4.2 Trustworthy AI & Ethics

• 4.3 Prototyping & Testing Tools

V. Learning & Community

This category of resources is the lifeline for staying at the forefront and solving problems.

• 5.1 Knowledge Acquisition

• 5.2 Community & Collaboration

VI. Infrastructure & Hardware

This category of resources focuses on the underlying foundation that supports AI computation.

• 6.1 Compute Resources

• 6.2 Optimization & Compilers

I. Model & Algorithm Core

1.1 Model Repositories/Platforms

Comprehensive Platforms

• Hugging Face - huggingface.co

  • Features: The world's largest AI open-source community and model hub, integrating models, datasets, and demos.

• ModelScope - modelscope.cn

  • Features: An AI model open-source community with Chinese characteristics led by Alibaba, focusing on Chinese models and applications.

• Kaggle Models - kaggle.com/models

  • Features: Affiliated with the well-known data science competition platform, making it easy to use with datasets and Notebooks.

Major Tech Platforms

• OpenAI Platform - platform.openai.com

  • Features: Provides official APIs and fine-tuning interfaces for top-tier closed-source models like the GPT series.

• Google AI Studio - aistudio.google.com

  • Features: Google's entry-level AI development platform, allowing for quick and free calls to the Gemini model API.

• Azure AI Model Catalog - azure.com/machine-learning/model-catalog

  • Features: An enterprise-grade model marketplace on Microsoft Azure, integrating numerous foundational models and customization tools.

• Amazon SageMaker JumpStart - aws.amazon.com/sagemaker/jumpstart

  • Features: An entry point on AWS for one-click deployment of a large number of pre-trained models and solutions.

Open Source Communities

• GitHub - github.com

  • Features: The world's largest code hosting platform, where most of the latest open-source AI models can be found through search.

• PyTorch Hub - pytorch.org/hub

  • Features: The official pre-trained model library provided by PyTorch, ensuring compatibility with the PyTorch ecosystem.

• TensorFlow Hub - tfhub.dev

  • Features: The official model library for TensorFlow, providing reusable machine learning modules.

1.2 Model Training & Fine-tuning

Training Frameworks

• PyTorch - pytorch.org

  • Features: Known for its dynamic computation graphs and ease of use, it is the mainstream framework for academic research and LLM training.

• TensorFlow - tensorflow.org

  • Features: An industrial-grade framework promoted by Google, with a powerful toolchain for production deployment.

• JAX - jax.readthedocs.io

  • Features: Designed for high-performance numerical computation, it has become an important tool for Google's internal and cutting-edge research when combined with Flax.

Fine-tuning Toolkits

• Hugging Face Transformers - huggingface.co/docs/transformers

  • Features: Provides a unified API for thousands of pre-trained models, making it the de facto standard library in the NLP field.

• PEFT - huggingface.co/docs/peft

  • Features: A parameter-efficient fine-tuning library from Hugging Face that significantly reduces the cost of fine-tuning large models.

• LoRA - github.com/microsoft/LoRA

  • Features: A mainstream PEFT method that achieves efficient fine-tuning through low-rank adapters and is widely integrated.

• LM-Switch - github.com/ContextualAI/lm-switch

  • Features: A toolkit for unified management and execution of multiple large models from different sources.

Distributed Training

• DeepSpeed - github.com/microsoft/DeepSpeed

  • Features: A deep learning optimization library from Microsoft, famous for its powerful ZeRO memory optimization technology.

• FairScale - github.com/facebookresearch/fairscale

  • Features: A PyTorch extension library from Meta, a collection of optimization techniques for large-scale training.

• PyTorch DDP - pytorch.org/docs/stable/distributed.html

  • Features: PyTorch's native distributed data-parallel training interface, which is stable and efficient.

1.3 Prompt Engineering & Optimization

Prompt Libraries

• Awesome Prompts - github.com/f/awesome-chatgpt-prompts

  • Features: A well-known collection of ChatGPT prompts, covering various roles and scenarios.

• Prompting Guide - promptingguide.ai

  • Features: A systematic prompt engineering tutorial website covering techniques, papers, and best practices.

Prompt Optimization Tools

• LangChain Hub - github.com/langchain-ai/langchain

  • Features: The prompt sharing platform for the LangChain framework, containing a large number of reusable chains and prompts.

• PromptPerfect - promptperfect.jina.ai

  • Features: An online AI prompt optimization tool that automatically improves prompt quality.

• Vellum - vellum.ai

  • Features: An LLM application development platform that integrates prompt debugging, version management, and deployment.

Chain-of-Thought Resources

• Related Papers, Blogs - arxiv.org, lilianweng.github.io 

  • Features: Access cutting-edge Chain-of-Thought (CoT) techniques by tracking top conference papers and well-known tech blogs.

1.4 Model Evaluation & Benchmarking

Evaluation Frameworks

• HELM - crfm.stanford.edu/helm

  • Features: A comprehensive language model evaluation framework from Stanford, with broad and systematic evaluation dimensions.

• Eleuther AI LM Evaluation Harness - github.com/EleutherAI/lm-evaluation-harness

  • Features: The de facto standard developed by EleutherAI, widely adopted by leaderboards like Hugging Face.

• TIDE - github.com/ElementAI/tide

  • Features: A fast error analysis framework for vision models (e.g., object detection).

Leaderboards

• Hugging Face Open LLM Leaderboard - huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

  • Features: Ranks open-source large models based on a unified standard, serving as an important reference for the community.

• LMSYS Chatbot Arena - chat.lmsys.org

  • Features: Truly reflects user preferences for large model chat capabilities through anonymous, random "blind test" voting.

Datasets

• MMLU - github.com/hendrycks/test

  • Features: A massive multi-task understanding test set covering 57 subjects, evaluating model breadth and depth.

• GSM8K - github.com/openai/grade-school-math

  • Features: A set of grade school math word problems released by OpenAI, specializing in evaluating model reasoning abilities.

• HumanEval - github.com/openai/human-eval

  • Features: A code generation evaluation dataset released by OpenAI, which tests model programming capabilities through test cases.

II. Software Development & Engineering

2.1 Development Frameworks & SDKs

AI Application Frameworks

• LangChain - langchain.com

  • Features: A leading framework for developing LLM-powered applications, emphasizing component composability.

• LlamaIndex - llamaindex.ai

  • Features: Specializes in providing efficient data access and Retrieval-Augmented Generation (RAG) applications for LLMs.

• Semantic Kernel - github.com/microsoft/semantic-kernel

  • Features: A lightweight SDK from Microsoft for deeply integrating traditional programming languages with AI model skills.

• Haystack - haystack.deepset.ai

  • Features: An end-to-end open-source LLM framework, with a special focus on building powerful search and question-answering systems.

Official SDKs

• OpenAI Python Library - github.com/openai/openai-python

  • Features: The official Python client for the OpenAI API, the standard way to call models like GPT.

• Anthropic SDK - github.com/anthropics/anthropic-sdk-python

  • Features: The official Python client library for the Anthropic Claude series of models.

• Google AI Python SDK - github.com/google/generative-ai-python

  • Features: The official Python SDK for calling the Google Gemini series of models.

Edge Computing Frameworks

• TensorFlow Lite - tensorflow.org/lite

  • Features: A lightweight solution from Google for deploying models on mobile and embedded devices.

• PyTorch Mobile - pytorch.org/mobile

  • Features: Provides an end-to-end workflow from training to deploying PyTorch models on mobile.

• ONNX Runtime - onnxruntime.ai

  • Features: A high-performance inference engine that supports running models in ONNX format on various hardware platforms.

2.2 Interfaces & Communication

API Gateways & Management

• Kong - konghq.com

  • Features: A high-performance, cloud-native open-source API gateway for managing and orchestrating microservice APIs.

• Apigee - cloud.google.com/apigee

  • Features: A fully managed API management platform under Google Cloud, providing complete API lifecycle management.

• Tyk - tyk.io

  • Features: An open-source, lightweight API gateway that supports REST, GraphQL, and gRPC traffic management.

Communication Protocols

• gRPC - grpc.io

  • Features: A high-performance, open-source RPC framework developed by Google, based on HTTP/2 and Protocol Buffers.

• WebSocket

  • Features: A protocol for full-duplex communication over a single TCP connection, suitable for real-time AI applications.

MCP-like

• Model Context Protocol - modelcontextprotocol.io

  • Features: An open protocol promoted by multiple companies to standardize the connection between AI applications and external data sources/tools.

2.3 Data Management & Processing

Data Acquisition

• Scrapy - scrapy.org

  • Features: A fast, high-level Python web crawling framework for large-scale scraping of structured data.

• BeautifulSoup - pypi.org/project/beautifulsoup4

  • Features: A Python library for parsing data from HTML and XML files, suitable for small-scale, quick extraction.

• Kaggle Datasets - kaggle.com/datasets

  • Features: A platform with a large number of community-uploaded data science datasets, often accompanied by competitions and code.

• UCI Machine Learning Repository - archive.ics.uci.edu

  • Features: A classic machine learning dataset archive widely used in education and research.

Data Annotation

• Label Studio - labelstud.io

  • Features: An open-source, multi-type data annotation tool that supports various tasks from computer vision to NLP.

• Scale AI - scale.com

  • Features: A platform providing high-quality data annotation and synthetic data services for enterprise-level AI.

• Prodigy - prodi.gy

  • Features: A scriptable annotation tool developed by the SpaCy team for efficient data labeling for machine learning models.

Vector Databases

• Pinecone - pinecone.io

  • Features: A fully managed vector database known for its ease of use and high-performance retrieval.

• Milvus - milvus.io

  • Features: A highly scalable open-source vector database designed for massive-scale vector similarity search.

• Weaviate - weaviate.io

  • Features: An open-source vector search engine with a GraphQL interface and hybrid search capabilities.

• Qdrant - qdrant.tech

  • Features: A high-performance, open-source vector database and similarity search engine written in Rust.

• Chroma - docs.trychroma.com

  • Features: A lightweight open-source vector database built for AI-native applications and rapid prototyping.

2.4 Deployment & Operations

Deployment Platforms

• AWS SageMaker - aws.amazon.com/sagemaker

  • Features: Amazon's end-to-end machine learning platform, offering a full suite of services from building to training and deployment.

• Google Vertex AI - cloud.google.com/vertex-ai

  • Features: Google Cloud's unified MLOps platform, helping users build and deploy models at scale.

• Azure Machine Learning - azure.microsoft.com/machine-learning

  • Features: Microsoft Azure's enterprise-grade ML lifecycle management platform, emphasizing security and integration.

• Vercel AI SDK - sdk.vercel.ai

  • Features: An SDK from Vercel for rapidly building and deploying AI-powered user interfaces at the edge.

Containerization

• Docker - docker.com

  • Features: The leading application containerization platform for packaging applications and dependencies into portable images.

• Kubernetes - kubernetes.io

  • Features: The industry-standard container orchestration system for automating the deployment, scaling, and management of containerized applications.

• KubeFlow - kubeflow.org

  • Features: An end-to-end MLOps platform built on Kubernetes, specifically for machine learning workflows.

Model Serving

• Triton Inference Server - github.com/triton-inference-server/server

  • Features: An open-source inference serving software from NVIDIA that supports high-performance deployment on multiple frameworks and GPUs.

• TensorFlow Serving - github.com/tensorflow/serving

  • Features: A high-performance serving system designed for TensorFlow models, suitable for production environments.

• TorchServe - pytorch.org/serve

  • Features: The official model serving framework from PyTorch, making it easy to deploy and scale PyTorch models.

Monitoring & Observability

• Prometheus - prometheus.io

  • Features: A powerful open-source systems monitoring and alerting toolkit, especially suited for time-series data.

• Grafana - grafana.com

  • Features: The leading open-source data visualization tool, often paired with Prometheus for monitoring dashboards.

• Weights & Biases - wandb.ai

  • Features: An MLOps platform for machine learning experiment tracking, dataset versioning, and model management.

• MLflow - mlflow.org

  • Features: An open-source machine learning lifecycle management platform covering experiments, code, models, and deployment.

III. Tools & Platforms

3.1 Model-as-a-Service (MaaS)

General-Purpose Large Models

• OpenAI GPT - openai.com/api

  • Features: Provides APIs for the world-leading GPT series of models, one of the de facto standards for industry applications.

• Anthropic Claude - anthropic.com

  • Features: A large model service known for its long context, strong security, and "Constitutional AI" principles.

• Google Gemini - deepmind.google/gemini

  • Features: Google's native multimodal large model, deeply integrated into its ecosystem, adept at understanding and generating various types of information.

• Meta Llama - llama.meta.com

  • Features: A leading open-source large model series released by Meta, allowing for self-deployment and commercial use by enterprises.

Vertical-Specific Models

• Biomedicine (e.g., AlphaFold) - alphafold.ebi.ac.uk

  • Features: A model developed by DeepMind that accurately predicts protein structures from amino acid sequences.

• Code (e.g., GitHub Copilot) - github.com/features/copilot

  • Features: An AI programming assistant powered by OpenAI technology that provides real-time code completion and suggestions.

• Image (e.g., Midjourney) - midjourney.com

  • Features: An AI image generation tool that interacts via Discord, known for its artistic and creative quality.

• Audio (e.g., ElevenLabs) - elevenlabs.io

  • Features: Provides industry-leading, highly expressive, and realistic AI voice synthesis and cloning services.

Open Source Model Hosting

• Replicate - replicate.com

  • Features: A simple cloud platform for running thousands of open-source AI models with one click, no environment configuration needed.

• Together AI - together.ai

  • Features: Provides powerful open-source model APIs and GPU compute, focusing on accelerating the open-source AI ecosystem.

• Banana Dev - banana.dev

  • Features: An inference platform focused on deploying and scaling open-source machine learning models with extremely low latency.

3.2 AI-Enabled Tools

Low-Code/No-Code Platforms

• Bubble - bubble.io

  • Features: A powerful visual programming tool for building complete web applications and integrating AI without code.

• Retool - retool.com

  • Features: A platform for quickly building internal tools and applications by dragging and dropping components, easily connecting to various AI APIs.

• Microsoft Power Platform - powerplatform.microsoft.com

  • Features: A low-code suite from Microsoft that allows enterprises to easily create AI assistants using tools like Power Virtual Agents.

Automation & Integration

• Zapier - zapier.com

  • Features: The world's leading automation tool for creating automated workflows between thousands of applications (including AI services).

• Make - make.com

  • Features: A powerful visual platform for building complex, multi-step automated integration scenarios.

• n8n - n8n.io

  • Features: A node-based, self-hostable open-source workflow automation tool, beloved by developers.

Task-Specific APIs

• Speech Recognition (e.g., OpenAI Whisper) - openai.com/research/whisper

  • Features: A powerful open-source speech recognition system from OpenAI that supports multilingual transcription and translation.

• Image Recognition (e.g., Clarifai) - clarifai.com

  • Features: An easy-to-use AI platform that provides APIs for computer vision and natural language processing.

• Document Processing (e.g., Adobe PDF Extract API) - developer.adobe.com/document-services

  • Features: High-fidelity parsing of PDF document structure, converting unstructured content into structured data.

3.3 Evaluation & Market Analysis Platforms

Model Rating Platforms

• LMSYS Chatbot Arena - chat.lmsys.org

  • Features: Reflects user preference rankings for large model chat capabilities through anonymous, random "blind test" voting.

• Hugging Face Open LLM Leaderboard - huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

  • Features: An authoritative leaderboard that systematically ranks open-source large models based on multiple academic benchmarks.

• Stanford CRFM - crfm.stanford.edu

  • Features: The evaluation platform of the Stanford Center for Research on Foundation Models (CRFM), which publishes in-depth evaluation reports like HELM.

Tool Rating Platforms

• G2 - g2.com

  • Features: The world's largest tech marketplace and software review platform, providing real user reviews and rankings.

• Gartner - gartner.com

  • Features: The world's most influential IT research and advisory company, whose Magic Quadrant reports are considered industry benchmarks.

• Capterra - capterra.com

  • Features: A review and comparison platform focused on helping small businesses find the right software.

Industry Reports & News

• AI Weekly - aiweekly.com

  • Features: A free weekly newsletter that curates the latest news, papers, and resources in the AI field.

• The Batch - deeplearning.ai/the-batch

  • Features: Produced by Andrew Ng's DeepLearning.AI, it provides easy-to-understand AI news and technical analysis.

  • (Note: The Batch is a publication of DeepLearning.AI, available via their website or email subscription)

• Sequoia Capital AI report - sequoiacap.com

  • Features: Trend insights and market analysis in the AI field published by top global venture capital firm Sequoia, highly forward-looking.