About this Video

• Video Title: GPT-5 is a freak
• Channel: AI Search
• Speakers: One unnamed speaker
• Duration: 00:35:53

Introduction

This video provides an unscripted review of GPT-5, showcasing its capabilities and limitations. The reviewer tests GPT-5's performance across various tasks, including coding, game development, physics simulation, image generation, and research, comparing its results with other leading AI models.

Key Takeaways

• GPT-5 excels in STEM subjects like coding, math, science, and research, generating complex code (e.g., beehive simulator, 3D racing game, fluid dynamics visualizer, ray tracing simulation) with fewer errors than other models.
• GPT-5 can create interactive applications like a CRM dashboard, a Photoshop clone, a meditation app, and a video effects editor, demonstrating its versatility in generating practical tools.
• GPT-5 shows proficiency in scientific research tasks, producing concise reports on e-commerce growth in Asia, medical research on Alexander disease, and sports medicine reports on ACL injuries. It also accurately identifies the location and event of a photograph.
• GPT-5 demonstrates agentic capabilities by autonomously generating a five-page storybook with images, although the image consistency needs improvement.
• While GPT-5 shows significant improvement in reducing hallucinations compared to previous models, its performance varies on different benchmarks, and the free version may offer a less powerful model.

Based solely on the provided transcript, here's a list of GPT-5's capabilities and limitations:

Capabilities:

• STEM Proficiency: Excels at coding, math, science, and research tasks. Successfully generated code for a beehive simulator, 3D racing game, fluid dynamics visualizer, and ray tracing simulation.
• App Creation: Created functional and interactive applications including a CRM dashboard, a Photoshop clone, a meditation app, and a video effects editor.
• Scientific Research & Synthesis: Produced concise reports on various topics including e-commerce growth in Asia, medical research (Alexander disease and ACL injuries), and synthesized information from web searches. Accurately identified the location and event from a photograph.
• Image Generation (with limitations): Demonstrated agentic capabilities by generating a five-page storybook with images, though image consistency across pages was inconsistent. Used the existing GPT-4 image model, not a new one.
• Reduced Hallucinations: Showed a significantly lower rate of hallucination compared to previous models, particularly in health-related questions.
• Coding Efficiency: Produced code with fewer errors than other top models tested.

Limitations:

• Image Generation Consistency: The images generated for the storybook lacked consistency in character depiction across pages. Relied on the GPT-4 image model.
• Minor Flaws in Some Outputs: Some generated outputs (taxonomy tree, physics course) contained minor visual or organizational flaws.
• Free Plan Limitations: The free plan may use a smaller, less powerful variant of GPT-5, resulting in lower quality responses.
• Benchmark Variations: Performance varied across different benchmarks, sometimes showing only marginal improvement over previous models. The benchmarks used were mostly those created by OpenAI.
• Lack of Explicit Model Selection (Free Plan): The free plan doesn't allow users to select the specific GPT model used.
• Black Box Nature: The internal workings of the model are proprietary and closed source, hindering a complete understanding of its processes.