Alibaba ACA GenAI cheat sheet

Qwen is Alibaba's family of large language models; variants include base, chat (instruction-tuned), code (Coder), and math-specialized releases.

Qwen model sizes span small edge-friendly models through very large dense and Mixture-of-Experts (MoE) models; MoE activates only a subset of parameters per token, lowering inference cost.

Qwen-VL models are multimodal: they accept image plus text input and produce text output; Qwen-Audio extends the family to speech/audio understanding.

Context window is the maximum number of tokens (prompt plus generated output) the model can attend to in one call; exceeding it requires truncation, chunking, or retrieval.

Tokenization splits text into subword units; token count is not equal to word count, and Chinese/CJK text typically consumes more tokens per character than English.

Temperature controls sampling randomness: low values (near 0) make output deterministic and factual; high values increase diversity. top_p (nucleus) restricts sampling to the smallest token set whose cumulative probability exceeds p.

Top-k sampling limits each step to the k most probable tokens; repetition penalty discourages reusing recent tokens.

Hallucination is fluent but unsupported output; grounding via retrieval-augmented generation (RAG) and instruction constraints reduce it.

Fine-tuning adapts a pretrained model to a domain or task by further training on labeled data; it differs from RAG, which supplies knowledge at inference time without weight changes.

Model Studio (Alibaba Cloud's generative AI platform, also called Bailian) provides hosted access to Qwen and third-party models through APIs, plus application-building tools.

API access uses an API key passed in the Authorization header; model selection is specified per request by model name.

The platform offers an OpenAI-compatible endpoint mode, so existing OpenAI-style client code can be repointed to Model Studio base URL and credentials.

Prompt templates separate fixed instructions from variable inputs; application orchestration chains prompt, model call, and post-processing steps.

RAG in Model Studio combines a knowledge base (document upload, chunking, embedding, vector index) with retrieval at query time, then injects retrieved chunks into the prompt.

Chunking strategy affects retrieval quality: chunks too large dilute relevance, chunks too small lose context; overlap preserves boundary information.

Embedding models convert text to vectors; similarity search (often cosine similarity) retrieves the nearest chunks for a query.

Plugins and tool/function calling let the model request external actions (search, APIs) whose results are returned to the model for a final answer.

Streaming responses return tokens incrementally to reduce perceived latency; token usage in the response is used for billing and quota tracking.

Platform for AI (PAI) is Alibaba Cloud's end-to-end machine learning platform covering data preparation, training, and model serving.

PAI-DSW (Data Science Workshop) is a managed notebook environment for interactive development with GPU/CPU instances.

PAI-DLC (Deep Learning Containers) runs distributed training jobs: users specify image, instance count, and instance type for a job.

PAI-EAS (Elastic Algorithm Service) hosts trained models as online inference services with autoscaling and versioned service deployment.

PAI-Designer provides a drag-and-drop visual pipeline with prebuilt components for data processing, training, and evaluation.

Model deployment requires an inference image or runtime compatible with the model framework (e.g., PyTorch, TensorFlow) and the model artifacts stored in OSS.

Object Storage Service (OSS) commonly holds training datasets and model checkpoints that PAI jobs read and write.

Scaling options include horizontal scaling (more instances) and vertical scaling (larger instance type); GPU memory must fit model weights plus activations.

Quantization reduces numeric precision (e.g., FP16 to INT8) to shrink memory and speed inference, trading some accuracy for efficiency.

Lingma is Alibaba Cloud's AI coding assistant, delivered as IDE extensions (e.g., VS Code, JetBrains) that provide inline completion and chat.

Core capabilities include code completion, natural-language-to-code generation, code explanation, unit test generation, and code review suggestions.

Lingma uses repository context (open files, related code) to ground suggestions; broader context improves relevance but raises latency and token use.

Agentic coding means the assistant plans multi-step changes, edits files, and runs commands, rather than only emitting single-line completions.

Enterprise deployment emphasizes data governance: code is not used to train public models, and enterprise tiers support private/isolated configurations.

An AI agent loop is: observe context, plan, call a tool (search, code execution, API), observe the result, and iterate until the goal is met.

Tool/function calling is the mechanism by which an LLM agent invokes external capabilities; the model outputs a structured call and the host executes it.

Human-in-the-loop review remains necessary: generated code must be reviewed and tested because models can produce plausible but incorrect logic.

Prompt design for coding agents should state language, framework, input/output contract, and constraints to reduce ambiguous generations.