Search - metacurate.io

An Introduction to the Transformer Architecture | Zenodo

An introduction to Transformer architectures, centered on the original model from Attention Is All You Need, starting with a high-level overview and then detailing its main components.

1 min read Listed 22 days ago in reddit/MachineLearning/Study Notes on Transformers and Hamiltonian Monte Carlo [P]

zenodo.org Tutorial

An Introduction to the Transformer Architecture | Zenodo

This document introduces Transformer architectures, focusing on the original model from Attention Is All You Need. It starts with a high-level view and then details the main components.

1 min read Listed 22 days ago in reddit/MachineLearning/Study Notes on Transformers and Hamiltonian Monte Carlo [P]

huggingface.co Documentation

Ultron28/ohlcv-transformer · Hugging Face

The document presents ULTRON, a 1.189B-parameter Financial Conformer for forecasting via 1-minute OHLCV data across 15 US assets, detailing training data, evaluation metrics, and reproducibility through audit logs; it includes technical results on drift, geometry, and execution policies, and notes dual publication formats by Suryaansh Prithvijit Singh.

7 mins read Listed 18 days ago in reddit/deeplearning/1.189B Financial Conformer: Architecture specification, multi-scale Inception tokenization, and why deep backbones freeze during online fine-tuning

x.com Blog post

Jungsoo Lee on X: "Energy-Based Models can now scale better than Diffusion models. Introducing TEM: Transformer Energy-based Models TLDR: • TEM outscales Diffusion without guidance • First scalable contrastive Energy-Based Models • Better generalization, test-time scaling, and OOD detection than… / X

TEM introduces Transformer Energy-Based Models, claiming they scale beyond diffusion models without guidance and offer scalable, contrastive EBMs with improved generalization, test-time performance, and OOD detection.

1 min read Listed 14 days ago in reddit/StableDiffusion/TEM: Transformer Energy-based Models

qlabs.sh Documentation

Quadrillion-parameter transformers and how to train them – Q

The document argues that current transformer sizes are far from optimal given compute; it presents a scaling framework suggesting quadrillion-parameter models trained with data-constrained, multi-epoch regimes, aided by snapshot ensembling, and it generalizes loss with factors for tokens, model size, epochs, and ensembling.

7 mins read Listed 6 days ago in Bluesky

dev.to Blog post

How Open-Source LLMs Differ: Comparing 130+ Model Architectures - DEV Community

A developer presents an interactive atlas comparing transformer designs across open-weight language models. It includes diagrams built from model configurations, a model comparison matrix, and a searchable gallery of 120 architectures.

8 mins read Listed 1 day ago in Dev.to

arxiv.org Documentation

[2502.09245] You Do Not Fully Utilize Transformer's Representation Capacity

This document introduces Layer-Integrated Memory (LIMe), a lightweight extension for Transformers that uses per-head, per-layer routing to incorporate representations from earlier layers via existing key-value buffers. LIMe improves efficiency and performance across tasks while preserving representation properties, with code available online.

1 min read Listed 16 days ago in reddit/LocalLLaMA/Researchers Discover a Way to Reduce LLM Training Compute by 15% Without Sacrificing Quality

huggingface.co Tools and repositories

Does Your Model Run on AWS Trainium? Look It Up

The article explains how TorchNeuron enables running many Transformers models on AWS Trainium, guiding users to look up model support in a Space, run tests locally with a specified stack, and interpret results from a Parquet dataset that records compatibility and performance details.

3 mins read Listed 22 days ago in Bluesky

x.com Tutorial

Şuayp Talha Kocabay on X: "I ran Qwen3.5-0.8B in pure Assembly. On an AMD EPYC 7B12, it generated 6.51 tokens/sec on one CPU thread. The Transformers reference reached 4.34. All 32 generated tokens matched in both runs. Inspired by @Arnesh_24’s gpt2.s. code: https://t.co/N4nf9vINNJ Tokenizatio… / X

A short-run benchmark shows Qwen3.5-0.8B implemented entirely in Assembly on an AMD EPYC CPU, achieving about 6.51 tokens/sec vs 4.34 tokens/sec for a reference, with identical 32-token outputs across two runs; tokenizer tables are prebuilt in Python, weights use BF16 loaded as FP32 by PyTorch kernels.

1 min read Listed 6 days ago in reddit/LocalLLaMA/I ran Qwen3.5-0.8B in pure Assembly

huggingface.co Tutorial

incoai/Qwen3.8-27B-DFlash2 at main

Guides to run incoai/Qwen3.8-27B-DFlash2 with multiple serving options (Transformers pipeline, vLLM, SGLang, Docker) and shows sample API calls to generate text from a prompt.

2 mins read Listed 7 days ago in reddit/LocalLLaMA/[Paper] DSpine - Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding - tencent