Misreading Chat

CUDA で書かれた PyTorch 用カーネルに森田が玉砕しました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• GitHub – Dao-AILab/flash-attention: Fast and memory-efficient exact attention
• GitHub – NVIDIA/apex: A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
• [2307.08691] FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• [2112.05682] Self-attention Does Not Need $O(n^2)$ Memory
• GitHub – tspeterkim/flash-attention-minimal: Flash Attention in ~100 lines of CUDA (forward pass only)

---

向井が画像生成の拡散モデルに入門しました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• Diffusion models from scratch
• [1503.03585] Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• [2006.11239] Denoising Diffusion Probabilistic Models

---

森田が飽きずに CUDA の教科書を読んでます。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

• Programming Massively Parallel Processors: A Hands-on Approach (Amazon.co.jp, Elsevier)
• NVIDIA H100 Tensor Core GPU Architecture Overview

---

Meta の vector search 実装の関連論文を向井が読みました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• facebookresearch/faiss: A library for efficient similarity search and clustering of dense vectors.
• Home · facebookresearch/faiss Wiki
• Optimized Product Quantization | IEEE Journals & Magazine | IEEE Xplore
• Product Quantization for Nearest Neighbor Search
• [1102.3828] Searching in one billion vectors: re-rank with source coding
• The inverted multi-index | IEEE Conference Publication | IEEE Xplore
• A Survey of Product Quantization

---

森田が CUDA の入門書を読み始めました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

---

• Programming Massively Parallel Processors: A Hands-on Approach (Amazon.co.jp, Elsevier)
• Evolution of the Graphics Processing Unit (GPU) | IEEE Journals & Magazine | IEEE Xplore
• David Kirk (scientist) – Wikipedia

AI 流行りで台頭中な Vector DB のサーベイを向井が眺めました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• [2309.11322] Vector database management systems: Fundamental concepts, use-cases, and current challenges
• [2310.11703] A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge
• [2310.14021] Survey of Vector Database Management Systems
  
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Databases in 2023: A Year in Review | OtterTune

---

Intel の CPU にやってきたニューラルなんとかの出自を森田がたどりました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• Always-on Vision Processing Unit for Mobile Applications | IEEE Journals & Magazine | IEEE Xplore (Google Scholar)
  
• Intel Core Ultra Ushers in the Age of the AI PC
• Intel® Movidius™ Vision Processing Units (VPUs)
• Amazon.com: Intel NCS2 Movidius Neural Compute Stick 2, Perfect for Deep Neural Network Applications (DNN) : Electronics
  
• Movidius – Wikipedia
• Very long instruction word – Wikipedia
• Single instruction, multiple data – Wikipedia
• Google’s Clips camera is powered by a tailor-made AI chip – The Verge
• The revolutionary chipmaker behind Google’s project Tango is now powering DJI’s autonomous drone – The Verge
  
• Qualcomm’s Hexagon DSP, and now, NPU – Chips and Cheese
  
• AMD XDNA™ Architecture
• AI Engine Technology
  
• Neural Engine | Apple Wiki | Fandom

---

LLM に解かせる難問集と採点結果を向井が睨みました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

---

• [2311.12983] GAIA: a benchmark for General AI Assistants
• gaia-benchmark/GAIA · Datasets at Hugging Face

---

(壊れた音声ファイルを配信してしまったので、修正したファイルで同じエピソードを再配信しています。ごめんね。教えてくれた皆さんありがとうございました。）

GPGPU ワナビーの森田が C++ 拡張の GPGPU 標準を眺めました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

• SYCL Overview – The Khronos Group Inc
• SYCL.tech – Find out the latest SYCL news, videos, learning materials and projects.
• GitHub – Tohoku-University-Takizawa-Lab/neoSYCL: A SYCL Implementation for CPU and SX-Aurora TSUBASA
• DPC++ Documentation — oneAPI DPC++ Compiler documentation
• GitHub – AdaptiveCpp/AdaptiveCpp: Implementation of SYCL and C++ standard parallelism for CPUs and GPUs from all vendors: The independent, community-driven compiler for C++-based heterogeneous programming models. Lets applications adapt themselves to all the hardware in the system – even at runtime!
• GitHub – triSYCL/triSYCL: Generic system-wide modern C++ for heterogeneous platforms with SYCL from Khronos Group
• Codeplay Software Ltd
• Intel Proposes Adding Full SYCL Programming Model Support To Upstream LLVM – Phoronix
• Migrate scripts to OpenGL ES 3.1  |  Android Developers
• Migrate scripts to Vulkan  |  Android Developers
• tensorflow/tensorflow/lite/delegates/gpu/README.md at master · tensorflow/tensorflow · GitHub

---

Meta 社のインターンが分散トレーシングで集めたログを睨んだ論文を向井が読みました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。

---

• Lifting the veil on Meta’s microservice architecture: Analyses of topology and request workflows | USENIX
• GitHub – facebook/fbthrift: Facebook’s branch of Apache Thrift, including a new C++ server.
• Canopy: An End-to-End Performance Tracing and Analysis System – Meta Research
• Yuri Shkuro
  
• Mastodon ID: misreading.chat@misreading.chat

---