Filter this collection + TopicsAll topics inference performance infrastructure optimization model deployment workflow integration rationale AI model category adoption model design and deployment seed fund sizing acquisition strategy venture fund strategy adapter layer design MCP connector approval connector architecture security review process MoE scaling efficiency numerical precision optimization training infrastructure performance content strategy public-private ecosystem design data center development realism infrastructure credibility stakeholder inclusion local acceptance data center development process product launch and pricing AI differentiation strategy market problem framing privacy and hardware design AI industry and gaming industry misalignment AI-native game design AI product design principle game revenue measurement AI misuse in game development fine-tuning efficacy AI alignment and principal-agent problems AI agent coordination design Agent delegation in coding tools Human vs. agent organizational constraints AI planning capabilities Multi-agent coordination web standards development test observability test automation design AI in software development standards development trust boundary design agent security risk performance-security tradeoff AI infrastructure security AI alignment and containment limits Empirical assessment of containment failure technical origin story distributed file systems Self-directed AI adoption AI-assisted software development AI compute allocation AI device strategy agent development usability agent adaptability pricing model AI marketplace integration customer case studies customer discovery case studies AI model capability comparison AI model performance and cost AI model reliability and token limits Application building iterations Coding task quality Assistant evaluation efficiency Customer support workflows coding agent reliability goal-directed AI product-market fit cloud storage pricing history S3 pricing history S3 current pricing AI products AI safety network effects of AI collaboration platforms AI agent collaboration challenges shared context in AI-augmented collaboration open science infrastructure developer tooling prediction uncertainty sales team development equity ownership in bootstrapped vs. funded startups founder sales capability reasoning modality post-transformer architecture motivation latent reasoning benefits AI infrastructure Enterprise AI communication design for AI AI team integration tool architecture limitation API design preference HTTP caching infrastructure CDN caching limitations AI-powered creator tools Algorithmic curation philosophy Content quality over AI origin AI model release multi-character TTS voice customization design thinking product complexity AI tooling AI research model deployment platform fine-tuning cost efficiency training duration AI infrastructure constraints qualification program design qualification scope limitation AI infrastructure justification AI developer education growth AI developer training scale Human advantages with AI interdisciplinary literacy Expertise and AI output quality Chip architecture Hardware flexibility AI infrastructure strategy AI search visibility Building companies AI and the economy AI intellectual property protection government data integration AI-assisted knowledge work evaluation latency tolerance in voice interactions session limits in voice agents voice agent function calling function selection accuracy media virality and AI narratives organizational safety practices AI safety discourse dynamics strategic AI adoption framework enterprise software sprawl problem identification gap AI-enabled tool creation open source sustainability open source licensing strategy license key distribution strategy AI evaluation criteria context layer efficacy enterprise AI architecture model distillation retrieval methodology static embedding performance AI search real-time voice AI latency budget perceived latency thresholds production latency SLOs release governance for AI agents incremental deployment automated agent validation personal agent security agent autonomy agent architecture Legal implications of attribution failure Causal attribution in AI outputs Exact deletion method for influence testing AI/SaaS product governance AI development paradigm AI implementation barrier iterative AI interaction prompt engineering for beginners AI tool onboarding runtime governance control AI governance assumptions AI supply chain attack vector AI governance sovereignty AI agent privilege escalation managed AI services trade-offs AI risk mitigation Decision making ontology design for AI agents agent validation workflow ontology-based business logic enforcement neurosymbolic AI architecture AI system security and observability workflow automation KernelBench GPU kernel evaluation agent system design failure analysis in AI systems Human–AI performance tradeoffs in RE-Bench MLE-bench offline Kaggle benchmark AI deployment architecture RE-Bench AI agent evaluation methodology AI evaluation and generalization model evaluation open science and AI impact scaling AI creativity and scientific discovery founder and creator agency Open-source software AI video workflow paradigm AI video-audio architecture AI video input flexibility AI agent evolution agent memory architecture enterprise adoption metrics runtime security design customer experience design product scope customer-centric metrics Focused teams Customer focus Leadership and feedback agent reliability threshold human-agent interaction developer experience AI adoption AI deployment strategy AI security AI governance AI personalization AI memory AI limitations in education AI research direction AI capability assessment AI research challenges PeopleAll people Dion Harris Tim Fernholz Jason Lemkin Chenghao Wang Kyle Wiggers Jennifer Ferro Josh Dzieza Nilay Patel Aisha Malik Piero Molino Ethan Mollick David Burns Olivia Johnston Matthew Green Peter Mattis 王吉平 王博 Eric Eyken-Sluyters Ruben Gamez Rob Walling Simon Willison Gabriel Grinberg Daniel Vogel Curtis Allen Abhinay Kathuria Daksh Gupta Kevin Hou Zach Lloyd Mike McCormick Anastasis Germanidis John Coogan Jordi Hays Ivan Zhou Anthony Costa Ross Andrew Paquette Zuzanna Stamirowska Nick Kuhn Santi Subotovsky Neal Mohan Maggie Appleton Mckenzie Lock Eric Nguyen Hassan El Mghari Vishal Ganeriwala Fatima Tambajang Wade Foster Matt Pocock Richard Ho Liam Dunne Ali Ghodsi Ben Moore Anton Leicht Aaron Levie Ben Gilbert Evangelos Simoudis Jose Nicholas Francisco John Schulman Nathan Lambert Demetrios Brinkmann Tibo Sottiaux Han Wang Benedict Evans Adam Stacoviak Katie Parrott Stephanie Baladi Ajeya Cotra Ben Rudolph Logan Markewich Parag Agrawal Sachi Shah Maarten Grootendorst James Grimmelmann Zheng Dai David Gifford Pedro Moura Kayla Cinnamon David Rosenthal Daniel Whitenack Chris Benson Kareem Amin Frank Coyle Yash Poojary Chris Pedregal Tarek Mansour Lilian Weng Zubin Pratap Demis Hassabis Jean-Baptiste Kempf shridharathi Jensen Huang João (Joe) Moura Jim Goetz Mark McLaughlin Nikesh Arora Markus Villig Roelof Botha Maya Hoffree Ilkka Paananen David Risher Abhishek Fatehpuria Dan Gallagher Dex Horthy Jim Swanson Dario Amodei Chris Olah Amanda Askell Luis von Ahn Pieter Levels SourcesAll sources How NVIDIA GPUs Help Accelerate OpenAI’s GPT-6 Astra Ultrafast Amazon releases its own Jev clone as decision models flood the web | TechCrunch 20VC x SaaStr: Anthropic’s S-1 Leaks, Instinct Hits $10 Billion in 33 Days, AMD Buys World Labs, and Meta Poaches MongoDB’s CEO How We Built Harvey’s Connector Library Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs Don’t be boring How Utah locals fought Kevin O’Leary’s AI data center Hearing tech startup Legato launches its AI hearing glasses | TechCrunch Piero Molino on AI in Games The Dot and the Swarm The State of Browser Testing - Software Engineering Daily Sidecars: A low-latency trust boundary for Sandboxes | Modal Blog Is sandboxing sufficient to contain rogue agents? Distributed databases with Peter Mattis When Glasses, Earbuds and Phones All Have AI, Who Will Unify Your “Second Brain”? | Snapdragon Summit 2026 Sierra joins the OpenAI Marketplace Episode 852 | Getting Big Name Customer Case Studies and More Listener Questions (with Ruben Gamez) Claude Sonnet 5.5 Customer testimonials published by Anthropic for Claude Sonnet 5.5 2026 in LLMs (so far) Comment: S3 Is the Future, S3 Is the Past AI-Generated Code Is Already Competing With Human Code — Daksh Gupta, Greptile Get Out of the Model's Way — Kevin Hou, Google DeepMind Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp Zero to One in AI Safety: Halcyon’s Mike McCormick on Launching 30 New Orgs and the Founder Bottleneck Runway’s WorldPrompt and the Engineering of Real-Time Worlds TBPN: John Coogan on personal agents and advertising Our Investment in Ando: Modern Messaging for Teams and Agents How Open Science Can Help Researchers Prepare for the Next Pandemic Bootstrapping From a $500K Goal to a $50M Company What Comes After Transformers? From AGENTS.md to Enterprise Deployment with Nick Kuhn Rethinking Communication for Human and AI Teams | Emergence Capital Comment: We just shipped support for the ugliest part of HTTP: Vary YouTube CEO Neal Mohan: Why We’re Betting On AI And Not Afraid Of It Tool: Gemini 3.8 TTS Playground Design Engineering with Maggie Appleton Parsing “Can” from “Should”: How Netflix's Product Team Got Thousands of Creatives to Trust an AI Overhaul 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) How to train your own Jev for $17 TBPN: John Coogan on agents, app interfaces, and advertising NVIDIA Launches DSX Ready to Qualify Power and Cooling Products for AI Factories From Enablement to Execution, Egypt’s AI Ecosystem Reaches Production Scale The Overhang No Code Is Code: Zapier CEO Wade Foster on Headless Tools, Zapier MCP & Automation Bench AI Skills with Matt Pocock Inside OpenAI’s First Chip How to get discovered in AI search Ali Ghodsi on learning as a CEO and evaluating management practices Anton Leicht on AI deployment, economic disruption and risk Box's Aaron Levie: On Reinventing Yourself in the AI Age and Enterprise Diffusion The Home Depot When Everyone Is 10x More Productive, No One Is Voice Agent Function Calling: Real-Time Data Lookup Guide AI researchers debate how close we are to recursive self-improvement One resignation turned the embers of AI fear into a wildfire Computer-Use Agents and the Future of the Agentic Internet Building Codex with Tibo Sottiaux Simpler, outcome-based AI pricing AI, tools and transformation — Benedict Evans Forking Cal.com to closed source (Changelog Interviews #685) Katie Parrott on using ChatGPT for writing and career reflection From enterprise search to enterprise context: what AI agents actually need Ajeya Cotra on agent persistence, coordination, and benchmark cheating Making Cities Awesome: Peregrine’s Nick Noone & Ben Rudolph Exploring Static Embedding Retrieval Parallel’s Parag Agrawal: Building a New Web for AI Agents Real-Time Voice AI Stack for Agents: Architecture Guide Release governance: guardrails for agents at scale An Agent Is Just an LLM in a For-Loop When AI art has no author: MIT study finds generated images often can't be traced to any training data | MIT CSAIL The People Who Understand Don't Build. The People Who Build Don't Understand. | CrewAI GitHub Copilot app for Beginners: Write your first prompt Disney: The Renaissance and the Empire Reconstructing how OpenAI agents attacked Hugging Face The Philosopher CEO | Clay Co-Founder Kareem Amin Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley Yash Poojary on automating growth experiments at Every Chris Pedregal on startups and work beyond meeting notes Kalshi’s Tarek Mansour vs. the Federal Government Harness Engineering for Self-Improvement Cartesia | A Guide to Choosing Voice AI Models The Walt Disney Company Demis Hassabis on Scientific Discovery and Personal Agency Jean-Baptiste Kempf on open-source code quality, maintenance and secure builds How to make remarkable videos with Seedance 2.0 – Replicate blog Jensen Huang on AI agent safety, leadership and trust Orchestrating Self-Evolving Agents with CrewAI and NVIDIA NemoClaw | CrewAI Palo Alto Networks ft Nir Zuk & Nikesh Arora - The Grudge That Transformed Cybersecurity Bolt ft. Markus Villig - From Bootstrapping in Estonia to a Global Leader in Mobility Supercell ft Ilkka Paananen - How an Early Pivot Led to ‘Clash of Clans’ and ‘Brawl Stars’ Lyft CEO David Risher on paying drivers more and the shift to robotaxis Trade Winds: The Rise, Reckoning and Reimagining of Vlad Tenev | Index Ventures 12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer Gen AI Present and Future: A Conversation with Jim Swanson, CIO at Johnson & Johnson | Greylock Dario Amodei, Amanda Askell & Chris Olah on AI Development and Safety Duolingo CEO Luis von Ahn thinks AI has a lot to teach us Pieter Levels on Bootstrapping Startups, AI Image Models, and Indie Development Runway Research | Introducing General World Models Publication yearAll dates 2026 2025 2024 2023 Filter this collection 1 people · 1 sources · 1 viewpoints
Reset filters IN CONTEXT
AI evaluation and generalization Choose a viewpoint. Follow it back to the conversation.
Lilian Weng Jul 4, 2026
Evolved harnesses generalize beyond their original benchmarks Read evidence ↗ IN CONTEXT AI evaluation and generalization 1 viewpoints
01 Evolved harnesses generalize beyond their original benchmarks Lilian Weng 2026-07-04 Equal sectors are reading positions, not rankings.
01 Evolved harnesses generalize beyond their original benchmarks Lilian Weng · 2026-07-04 Showing 1–1 of 1 viewpoints · Newest sources first
Previous 1 / 1 Next
Selected viewpoint
Evolved harnesses generalize beyond their original benchmarks A harness evolved on Terminal-Bench-2 transferred without further evolution to SWE-bench-verified, indicating it encodes general engineering experience in harness components rather than benchmark-specific optimization.
These are individual perspectives, not a measure of consensus. Source material stays in its original language.
Supporting evidence
Lilian Weng · Jul 4, 2026
Harness Engineering for Self-Improvement Original excerpt
On Terminal-Bench-2, AHE achieved better than human-designed harness (OpenCode, Terminus-2, Codex) except for Hard tier and a few other self-evolve baselines (ACE, TF-GRPO). The same frozen harness, without further evolving, transfers to SWE-bench-verified, indicating that the evolved harness is able to encode engineering experience into harness components rather than doing benchmark-specific optimization. Source & context
Publication dates describe the sources, not changes in belief.