Yesterday, I dropped a new video on YouTube breaking down the new considerations you should be taking when using GPT-5. We talked about the changes OpenAI implemented with GPT-5 and I provide a high-level strategy for getting truly professional output from the model using the MISTAKES framework.

If you prefer to watch or want the full context, you can see the deep dive right here:

In the video, I mentioned that I created a detailed companion guide with specific, copy-and-paste prompts and checklists.

This is that guide.

For those who are ready to go from theory to practice, this is your actionable toolkit.

Subscribe now

The GPT-5 "M.I.S.T.A.K.E.S." Field Guide

Below is a breakdown of a specific mistake and the specific prompt or fix you can use immediately.

As I explained in the video, don’t be afraid to ask the model to work harder if that is you what you need.

THINK HARD

M — Map

• Mistake: Jumping in without a mission brief (goal, audience, constraints, success criteria).

• The Fix: Give GPT-5 a full mission brief before you start.

Role: [expert]. Goal: [outcome]. Audience: [who]. Inputs: [links/notes]. Constraints: [style, length, compliance]. Success: [acceptance tests/metrics]. Output format: [sections/table/JSON]. Only proceed if unambiguous; otherwise ask 3 clarifying questions.

Mistake: Treating GPT-5 like a search box.

The Fix: Ask for a plan first, then the deliverable.

Before drafting, outline the approach, assumptions, and risks. Then wait for my ‘go’.

I — Investigate

• Mistake: Blind trust in facts, dates, and numbers.

• The Fix: Require verification paths for every claim.

Mark each claim High/Med/Low confidence; list what would change the answer; propose 2 ways to verify.

Mistake: Letting your own prompt bias anchor the answer.

The Fix: Ask for both the solution and rebuttal.

Generate the answer and a counter-answer that would be true if my premise is wrong; compare them.

S — Sketch

• Mistake: One-shot mega-prompts that try to do everything at once.

• The Fix: Scaffold your request. Go from brief → outline → sample → full draft. Lock in the structure early.

Propose 2 alternative outlines with tradeoffs; I’ll pick

• Mistake: Over/under-steering the router.

  • Over: Forcing “think hard” on trivial tasks → slow + verbose.

  • Under: Skimming complex tasks.

• The Fix:

  • Prompt tag for depth:

    This is complex, reason step-by-step and show uncertainties.

  • Prompt tag for speed:

    This is routine, answer concisely, no deliberation transcript, max 6 bullets.

T — Test

• Mistake: Accepting the first good-looking output.

• Fix: A/B by instruction delta.

Produce two distinct approaches with rationale and failure modes. Then recommend one.

• Mistake: Accepting the first good-looking output without challenge.

• The Fix: Red-team your own output to find the hidden flaws.

Stress-test this plan: adversarial inputs, worst-case constraints, ethical/compliance pitfalls. Output: issues → mitigations.

A — Adapt

• Mistake: Using static prompts that don’t learn from previous outcomes.

• The Fix: Close the feedback loop. Use GPT-5 to improve its own instructions.

Here is feedback from use (wins/gaps). Rewrite the prompt as v2; diff the changes and explain expected impact.

K — Kickstart

• Mistake: Re-solving the same tasks from scratch every time.

• The Fix: Build a personal library of your best prompts, schemas, and starter packs for each workflow.
  

  BONUS TIP: For each workflow create a scheduled run in ChatGPT to automatically run every day so it’s already done when you are ready to look at it.

E — Evaluate

• Mistake: Judging the output based on "vibes".

• The Fix: Define your acceptance criteria first. Have "Done when" checks for every brief.

Before output, self-check against these 5 acceptance tests. If any fail, revise and note what changed.

S — Sustain

• Mistake: Long, rambling chats that accumulate stale context and confuse the model.

• The Fix: Start fresh sessions for each new project phase. Paste only the minimum brief and the most current artifacts to keep the context clean.

Your Drop-in Prompt Stubs

Here are some of the most powerful, reusable stubs from the guide that you can paste into your work today.

The Universal Mission Brief:

Role: [expert]
Goal: [clear outcome]
Audience: [who]
Inputs: [links/notes]
Constraints: [style/length/policy]
Acceptance tests: [1..5]
Output: [format/schema]
First: ask 3 clarifying questions if needed.

For Critical Self-Review:

Run a preflight: coverage, clarity, correctness, compliance. If any <4/5, revise and say what changed.

For Red-Teaming & Skepticism:

Critique this as a skeptic: edge cases, failure modes, bias. Propose mitigations with effort/impact.

Your 5-Minute Pre-Flight Check

Before you start any major task with GPT-5 that has to be right, run through this quick audit:

• Do we have a mission brief with acceptance tests?

• Is the format pinned (e.g., a schema, table, or specific section layout)?

• Have we A/B tested at least one alternative approach?

• Are there verification steps for critical claims explicit?

• Did we document the final prompt and inputs for reuse?

I sincerely hope this guide not only helps you avoid some frustration with the new model so you get truly professional and impactful output from GPT-5.

Which of these fixes will you implement first to enhance your workflow? Hit reply and let me know…I read every email.

Best,

Daniel

P.S. I’m working on a new format that pairs with the broadcast content and comes out more frequently. Let me know if you like where this is going.

Conversely, please do also let me know if the previous mega-pack version I have been doing monthly is still valuable and if you prefer that.

Did I make a MISTAKE with the new approach? Let’s find out!

There might be room for both, but I want to hear from you either way! And if you can’t pick one, just hit “reply” and tell me directly.

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

Subscribe now

My Thoughts

Technology. History. Travel.

These three things all have ‘experience’ in common.

Technology creates a user experience.

History allows us to see the world through the eyes of a time not our own.

And Travel enables us to experience places we otherwise would not have seen.

This month, I’ve had a chance to ‘experience’ all three in a completely new way.

I was incredibly blessed to travel to Europe with my family, and it wouldn’t be AI MISTAKES if I didn’t bring along some AI to enjoy the trip fully!

Before we dive into the headlines, I'd like to share some brief observations with you. I’ve seen the world, but also in some new and unexpected ways. With the power of AI, I’m able to take you with me in more ways than one!

For this trip, I was able to connect with AI in 3 special ways:

1. Meta Ray-Ban AI-enabled glasses

2. Replit AI coding

3. AI in my pocket to fill in all the places that still needed more insights.

No buyer’s remorse on the glasses, but not because the AI wowed me. It was the freedom from screens that made the difference.” More details on that later (subscribe to the YouTube channel to see my review and deeper coverage of the Meta Ray-Ban glasses), but I must tell you that I was able to see and experience the trip without suffering behind an LCD screen viewfinder, which for me was a fantastic benefit!

Now, for the second way, I have to share with you that every time I take a trip, I nearly always lament that I haven’t fully captured the time away because I neglected to record my experiences. The good news is that we can now code our own journaling app, perfectly tailored for the trip! The bad news is, you still have to be intentional about putting things into the journal.

And for the third way, I think it’s fair to say that being a tour guide is about to get more interesting. For much of our tourist visits, we benefited from a paid guide. That makes for a more expensive trip, but you come away with far more than you expect, as you gain better insights and details that you wouldn't have obtained on your own. But a funny thing happened along the way. The added details the guides provided only served to whet my appetite for information, and my handy AI apps (Perplexity and ChatGPT) were there to fill out my experience with much more detail.

The good news is that what you take away from travel stands to improve substantially over the next few years. The bad news is that there is no excuse for a bad guided tour now, because, honestly, anyone should be able to gather the necessary contextual information to have a completely informed experience. In the few places I tried it, snapping a picture of a monument or cityscape nearly always provided exactly the supplemental information I was looking for, and often without much more than a quick snapshot and a question prompting the information I was seeking.

All in all, it was a fantastic trip, made all the better by taking the world’s information along so that I could fully appreciate all that we saw.

And speaking of seeing with AI, the most fantastic part is that all of it was pretty low effort’s that led to an enriching experience for my family and I along the way.

One of the most exciting moments for me was generating the image above of the Ancient Theatre of Taormina in Taormina, Sicily, Italy.

Below is the wide shot I took (on the iPhone from the top of the location) myself while visiting. However, while the tour guide was explaining the environment, I thought, “I wonder what a graphic rendering would look like if this were on the History Channel?”

Admittedly, the generated image was a one-shot with ChatGPT’s GPT-1 image model, but I didn’t exactly go out of my way to make the prompt. I just used the “valorum covering” context that the tour guide provided.

The Roman theater at Taormina complete render of ancient architecture complete with Valorium covering 

It captured all the key details fairly well, and it took only about a minute. It certainly doesn’t have anything on the actual, detailed work that researchers put together for a highly researched and detailed version. Still, it was a great way to augment what I was experiencing in person at that moment.

We had many great experiences, but one in particular was had while in harbor at Palma de Mallorca, Spain. After we’d returned from our tour of the city, and while vibe-coding more apps with Replit while sitting on the balcony, I noted a half-dozen incredible yachts in the harbor.

My curiosity got the best of me, so while still wearing my Meta glasses, I asked, “Hey Meta, tell me about these yachts over here.”

I got the rather disappointing reply:

There are several yachts docked in the marina, with a variety of sizes and styles. Some have sleek, modern designs, while others appear more traditional or classic.

I did not pursue the matter any further with the embedded Llama model. So, then I asked ChatGPT. Yeah, I had a pretty good idea that one of the Yachts was for Mark Zuckerberg and another for Jeff Bezos. (See the ChatGPT conversation here.) It was inconclusive and used the images for location context rather than ship recognition. Still, honestly, I was impressed with the level of details provided by ChatGPT 4o, and it was a simple prompt along with two pictures:

“What can you tell me about these yachts” 

Granted, it’s still not fair to put an on-device model up against the giant GPT-4o, but it just goes to show how far we’ve come in only a few years.

So as you’re traveling to and from with your friends and family to places near and far, take a moment and ask the AI in your pocket about what you see, you never know when it might take you to a whole new place of understanding the world.

Notable Headlines

Google

• Gemini CLI (This page details Gemini models and their capabilities, including CLI access): Google introduced Gemini CLI, a free AI tool that brings Gemini's coding and research powers directly into developers' terminals.

• Gemma 3n: Google is opening up access to Gemma 3n, a mobile-first model for building on-device apps across text, images, audio, and video.

• Android XR (General AI updates, often includes mentions of new hardware integrations): Google unveiled new smart glasses powered by Android XR and integrated with the Gemini model family, offering real-time captions, directions, translations, and a personal assistant.

OpenAI

• o3-pro and o4-mini: OpenAI released o3-pro, their latest "reasoning-first" model, and o4-mini, a smaller, cost-efficient reasoning model.

• AI Hardware Device: OpenAI, in collaboration with Jony Ive’s startup io, is reportedly working on a portable AI device, hinting at OpenAI’s hardware ambitions. **UPDATE: The page has been removed, so I’ve linked to an alternative article.

Anthropic

• Claude Code: Anthropic launched Claude Code, a coding agent previously offered as a research preview, along with a Claude Code software development kit.

• Project Vend: Anthropic conducted an experiment called Project Vend, where Claude Sonnet 3.7 ran a vending shop, demonstrating limitations in economic tasks requiring persistence, judgment, and autonomy.

• AI Fluency Course: Anthropic launched a new AI Fluency course designed to help individuals collaborate with AI systems effectively, efficiently, ethically, and safely.

• Apple Considers OpenAI/Anthropic for Siri: Apple is reportedly considering using AI technology from OpenAI or Anthropic to power its digital assistant Siri. **UPDATE: Just before publication, posts on X/Twitter indicated they have selected Anthropic

Meta

• Meta's Offers $100 Million Sign-On Bonuses to Lure OpenAI Talent: Meta is shaking up the AI world with eye-popping $100 million sign-on bonuses, all in an ambitious bid to woo top talent from OpenAI.

• Oakley Meta Glasses Announced: Meta and Oakley are collaborating to deliver a new category of “Performance AI glasses”.

Learning, Tools, and Experiments

This month, the best experiments are in CLI. Both Google and Anthropic dropped tools that help you code. Use the news links to get into Google’s Gemini CLI for free, and Anthropic’s Claude Code will cost you a subscription (or at least a sign-up for a free trial).

I can also recommend a past tool I shared called Warp, which is an AI-enabled CLI or Terminal app.

AI MISTAKES

Even as AI opens our eyes to new possibilities, it sometimes sees the world wrong, and those mistakes can be illuminating too. A case in point: an AI-written travel article infamously recommended tourists visit the “beautiful” Ottawa Food Bank when exploring Canada’s capital, euronews.com. Yikes! That’s not the kind of attraction either locals or tourists want from a tour guide!

The episode was a reminder that while AI can compile facts, it doesn’t grasp nuance or appropriateness like a human does. In this case, the algorithm knew the food bank existed and had high “interest,” but lacked the common sense to realize it’s not a tourist attraction.

On a different front, AI image generators have been busy reimagining historical scenes – sometimes too convincingly.

Online communities have been captivated by a wave of black-and-white “photos” depicting historical events: a mother and child during the Great Depression, soldiers in World War II, and ancient wonders of the world. The problem is that many of these viral images are complete AI fabrications, theglobepost.com.

What can we learn from these missteps?

First, AI doesn’t intend to get things wrong, but it will confidently present false or context-blind outputs if we’re not careful.

Travel AIs don’t have lived experience; they juggle data.

Vision AIs don’t know the truth; they pattern-match pixels.

As end-users, we have to provide the missing judgment. Double-check the recommendations before following them blindly, especially if you're using them to navigate a tourist spot. We received numerous advisories from our tour guide to be cautious of pickpockets and people who are less than welcoming to tourists.

Use AI as an aid, but still rely on traditional maps and information to guide your path.

In short, keeping humans in the loop is key.

AI can help us see differently, but we must not close our eyes to its flaws.

After all, making MISTAKES is a fundamental part of learning, for humans and AIs alike. The goal isn’t to avoid all errors (impossible), but to catch them, understand them, and build better tools as a result. By doing so, we ensure that AI truly augments our reality, rather than distorting it.

Let’s make July the month to see the world from a new perspective!

Safe travels, and stay curious!

Go see the world and use AI to help you navigate it and share new experiences the world over. And make some meaningful AI MISTAKES along the way.

If you’re enjoying my newsletter, I'd appreciate it if you could share it with someone who could benefit from this valuable information.

Share AI MISTAKES

Credit: NASA

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

My Thoughts

On May 25, 1961, President John F. Kennedy stood before Congress and proposed that the US "should commit itself to achieving the goal, before this decade is out, of landing a man on the Moon and returning him safely to the Earth."

This was an incredible challenge, met with an even more astonishing effort to bring it to reality.

In less than 10 years, on July 24, 1969, the Apollo 11 mission splashed down in the Pacific Ocean, answering the President's challenge. It’s the story of engineering prowess, scientific exploration, and man's triumph over incredible challenges.

We worked like mad to make it happen. We put 12 men on the moon.

And then we stopped.

We never went back.

It’s been over 50 years.

We lost the vision.

I recently watched a documentary about the Apollo missions, released in 2019, When We Were Apollo.

Near the end of the documentary, one of the people interviewed explained the problem:

We had “no good vision of what to do next.”

Ok, I hear you saying, “What does all this have to do with AI?”

Vision is everything.

The will to act is what drives us, and AI is becoming increasingly capable of the "action” part.

As humans, we have the distinct ability to dream. In fact, I would argue it is not only an ability, but an obligation.

While we have given computers the ability to speak and do things, the dream still has to come first.

Just as the Moon landing inspired a generation to look upward and…

AI has the potential to captivate and enable minds. To inspire the next wave of thinkers and creators.

It’s making things that were only dreams of the past not only possible but real.

So, the question I have for you is:

“What is your DREAM?”

This question is more complicated than you think. It will likely take time to answer this question, but you must act once you know the answer.

Once you read this, my challenge for you is to act.

What action must you take? Speak your dream into existence.

What would happen if you spoke that dream into a session with ChatGPT (or Claude, or Grok, or Meta, because they all talk back now, too)?

I know, it still seems a little crazy to just openly talk to a computer. It still feels a bit strange even to write this idea here.

Talk to an AI. Share your dream with advanced AI like Anthropic’s Claude 4 or OpenAI’s o3.

Speaking about your dream changes your mind, but speaking about it, even to an AI, could help you realize your dream.

The sooner you speak it, the sooner you can refine it and iterate with rapid feedback.

But don’t stop at speaking. With the latest AI capabilities from the likes of ElevenLabs, Midjourney, RunwayML, and the new Veo3, you can not only see your dream in text, but also hear it, see it, and watch it.

This is the superpower that AI gives us.

If you’re worried someone will see your mistakes, worry no more.

Let go of the fear of MISTAKES and start making progress today.

Use the MISTAKES framework as well, for help.

And if at any stage, after you recognize and get clarity on your dream, you need someone to listen, I would love to hear from you and cheer you on.

Until then, here’s more on what’s notable in AI that might help kindle your dreams.

Notable Headlines

• Google Gemini 2.5 Pro
  Google's latest Gemini model upgrades interactivity, code generation, and workflow automation.

• Veo3 from Google
  Google’s new Veo3 elevates text-to-video generation with cinematic fidelity and fast rendering for creators and studios.

• Claude 4 by Anthropic
  Claude 4 brings long-context reasoning and strong tool use to the forefront, raising the bar for enterprise-grade AI performance.

• OpenAI Codex
  OpenAI announces a major overhaul to Codex, integrating newer models with tighter IDE support and team productivity features.

• OpenAI Retiring GPT-4.5 in ‘months’
  OpenAI has shared they intend to deprecate GPT-4.5, consolidating its offerings around GPT-4o and the forthcoming GPT-5 lineup.

• Meta AI Hits 1B Users
  Meta's AI assistant sees massive adoption across its ecosystem, including WhatsApp, Instagram, and Messenger.

• Opera Neon: The AI Browser
  Opera launches an "agentic browser" that can research, automate tasks, and even code.

• Anthropic's AI for Science Program
  Claude is being used to accelerate discovery in biology and life sciences with a new research partnership program.

• DeepSeek R1 Model Updates
  DeepSeek releases an update to R1. No news on a ‘cleaned-up’ western version yet.

• Perplexity Labs Debuts
  A new workspace from Perplexity turns simple queries into rich reports, dashboards, and apps.

• Godela for Engineering Design
  This physics-trained AI model solves real-world engineering problems with unmatched speed and accuracy. You really can ‘build’ with AI now.

Learning, Tools, and Experiments

If you haven’t yet, speak your idea into one of these tools and build something, even small.

1. ChatGPT Deep Research

   Auto-generates PDF research reports with citations—perfect for business briefings. This is great for getting a quick document, and I also use it to accelerate working across multiple AI tools like Anthropic, Grok, and Meta AI.

2. Veo3 – Google’s Cinematic Video Model

   Generates high-quality videos from text prompts with cinematic style and advanced control. I’m collaborating with an industry insider for an upcoming YouTube video. (Don’t forget to subscribe!)

3. Claude 4.0

   Anthropic dropped new models and added voice mode. Go try it out!

A Few Tools I Regularly Use

Some of you have asked me about the tools I regularly use to share content and manage my work:

Descript: AI Video Editor
Perplexity Pro: My default go to for any question and some serious research
Gamma.app: Quickly make a presentation on any topic with ease.
Typeset: Build presentations, documents, social media content.
StreamYard: Live Video Streaming to multiple platforms.

Keep Learning

A comment on X (formerly Twitter) I saw earlier this month reminded me of this:

There are many prompt courses available, and I even created a video on “Is Prompt Engineering Dead?” last year; however, there is still value to be gained from learning this core skill.

If you aren’t using DSPy, and you haven’t already adopted one of the methods I shared in my prompt engineering presentation from 2023, you can pick up one of the courses offered by Microsoft, Anthropic, or OpenAI.

If you spend a significant amount of time using AI tools, it never hurts to step up your game to ensure you are using good-quality input.

Subscribe now

AI MISTAKES

Any time we push the boundaries of what’s possible, we haven’t truly done the work well until we reach a point of failure. There were some real AI breakthroughs over the last month, but there were also some real issues.

At the end of April, OpenAI updated their latest GPT model, and suddenly, it seemed everyone was complaining about how it seemed to be choosing agreement over accuracy.

A few days later, OpenAI admitted that their latest model updates included some AI MISTAKES.

In a May 2 post, Expanding on what we missed with sycophancy, they shared a bit more detail on what happened and what was learned.

I applaud the OpenAI team for recognizing the problem, taking steps to reverse the change, and then publicly sharing what they’ve learned.

Remember the MISTAKES framework:

Map, Investigate, Sketch, Test, Adapt, Kickstart, Evaluate, Sustain

This was a real-world example of what the ‘E’ and ‘S’ in the MISTAKES framework are all about: Evaluate what went wrong, and Sustain the learning publicly.

This situation has elevated my respect for how OpenAI has been operating after the Sam Altman leadership challenges last year, and I was pleased to see how they handled it.

From my perspective, all the frontier model builders have improved in how they handle model behavior problems over the last year. However, they must also continue to share publicly what they are learning. This is both the principle of sound engineering and the value of AI MISTAKES.

Let’s make June the month of bold vision, shared dreams, and forward action.

Go speak your dream. Build with AI. And make some meaningful AI MISTAKES.

If you’re enjoying my newsletter, do me a great favor and share it with someone that could use this valuable information for their own benefit.

Share

As we close out the first quarter of 2025, we’re already seeing a fascinating shift in focus. New methods from training algorithms, using the <thinking> tag, or simply giving a model time to *think* before answering, are here to stay. Reasoning capabilities are among the defining frontiers in generative AI, with major players and open-source communities pushing boundaries in how models think, act, and interact. We’re not going back.

Can you believe that it wasn’t all that long ago that we were saying “AI can do amazing things, but we have no idea how it arrives at the answer we get.” Those days already seem long ago now, especially with Anthropic’s research sharing recent revelations about what happens inside the black box, and this change is one that I particularly enjoy. Why? Because I think showing how the model arrives at an answer is powerful and instructive, and it’s exciting that we’re making progress in reaching explainability. However, I can also admit that there’s far more to learn!

Over the last few weeks, we’ve seen service after service reveal more about what’s happening with the model as the responses are generated. These ‘thinking models’ and circuit tracing approaches are fun and bear more time and attention. It’s been a hectic month of model updates, new features, and incredible new capabilities. I’ve hardly even had time to put down my use of Deep Research across Gemini, OpenAI, and Perplexity, and these same providers have all released new features for image generation (Google & OpenAI, I) haven’t been able to put it down. There are no excuses for not trying these services; many are entirely free! (Google is making everything accessible in AI Studio, and OpenAI has already shared plans to make the new GPT-4o and image generation available at no cost (though limitations apply, you have at least enough to experiment and make AI MISTAKES!

This is a fundamental shift. Until recently, AI models have been black boxes, and they’ve been held beyond payment limitations. So far in 2025, we now have a way to not only understand how the model came up with the answer, but can also use that to break down a problem, learn from it, and even correct the approach in future prompts. It’s incredible what’s already possible in 2025!

The significance of this cannot be overstated. AI has long been about prediction, but now it’s about understanding, explanation, and exploration.

Notable Headlines

Anthropic

• Claude 3.7: Claude 3.7 improves performance in coding, math, and reasoning tasks, while reducing hallucinations and increasing speed.

• MCP: The Model Context Protocol (MCP) is Anthropic's open protocol for model-to-model and tool-to-model communication, designed for future agentic systems.

Google

• Gemini 2.0 Flash: A lightweight, efficient version of Google's Gemini 2.0 model, optimized for cost-effective performance and fast response times.

• Gemini 2.5: The latest iteration of the Gemini model family, showcasing advancements in reasoning and multi-modal performance.

• Gemma 3: An open model developed by Google DeepMind as part of their push for responsible open-source AI.

• TxGemma: An open-source variant of Gemma fine-tuned for biomedical and therapeutics-related tasks.

OpenAI

• o3-mini: A compact and efficient model optimized for low-latency tasks and edge deployments.

• GPT 4.5: A transitional model between GPT-4 and GPT-5, offering improved speed and reasoning capabilities.

• GPT-4o: An omni-modal model that processes text, image, and audio simultaneously for richer contextual understanding.

• GPT-4o-transcribe: A next-gen audio model capable of near real-time transcription and translation with high accuracy.

Microsoft

• Think Deeper in Copilot: Introduces the "Think Deeper" capability in Microsoft 365 Copilot using OpenAI’s o3-mini model for enhanced reasoning, faster performance, and visual enhancements.

• Copilot Release Notes - March 2025: Details new features, including Copilot availability on GroupMe and Viber, and product price tracking.

• Copilot Vision Preview: Announces the preview release of Copilot Vision, enabling camera-based real-time assistance, especially for Android users in the U.S.

Perplexity

• Deep Research: A new feature that enhances Perplexity’s research capabilities by surfacing citations and deeper source validation.

• Cost Optimization of Sonar: An improvement to the Sonar model, offering better performance at a lower cost than some OpenAI offerings.

Learning Tools and Experiments

OpenAI Responses API + Agents SDK

https://platform.openai.com/docs/guides/agents-sdk

Cloudflare Agents

https://developers.cloudflare.com/agents/

OWL: built on CAMEL-AI Framwork, improve your agent interactions https://github.com/camel-ai/owl

MCP servers

https://github.com/modelcontextprotocol/servers

Mistral OCR

https://mistral.ai/fr/news/mistral-ocr

AI MISTAKES: Learning from the Past, Preparing for the Future

In the introduction to this month’s newsletter, we’ve talked about reasoning and shared the paper that Anthropic recently published, showing off the experiments that they conducted to better understand what’s happening inside the model.

It’s now increasingly clear that learning from mistakes will be a part of either the model-building or the mechanisms that control how a model answers questions.

The reality is that LLMs are trained through a multi-stage process that allows them to “learn” from mistakes and improve their responses. What we’ve historically done during the training process is drive improvements. These are not in real-time during a conversation but instead by using techniques such as self-supervised pretraining and reinforcement learning from human feedback (RLHF).

One of the significant research breakthroughs coming from many organizations trying out different techniques is that we can learn from what works and what doesn’t work in how we train (and use the models). The people behind DeepSeek have shared that they got great results just by taking the RL and skipping the HF.

Reinforcement Learning without Human Feedback (RL)

The core innovation was to replace the human feedback mechanism with rule-based rewards. This means that the model training relied on a set of automated trial and error optimizations. This took far less time but still led to good results.

By replacing the human feedback mechanism, they found that it could be a fast responding mechanism that took far less effort and time to train. The trade-off, however, was having to develop an algorithm that dealt with logical coherence, task completeness, and output fluency.

The best part of the outcomes from this experimentation was the discovery of yet more so-called “emergent properties” like the complex reasoning behaviors (chain-of-thought and self-correction) that I discussed in the introduction. Still, they were not explicitly asked for… it happened as a byproduct of moving to the new method.

It’s still necessary to leverage human feedback and give guidance and instruction to the model. Still, it happens at different times in the development cycle. It allows a dramatically accelerated schedule and better results without requiring substantial annotation teams to instruct the model development.

I can’t help but think that as we continue to advance AI models, we will make them ever-so-much more like us. This can only mean one thing: mistakes are here to stay, so we might as well get started.

Now make some mistakes by experimenting and learning with AI!

Introduction: From 2024 to 2025 – A Moment of Transition

Welcome to 2025! The future holds both the promise of innovation and the lessons of the past. As we stand on the threshold of 2025, let’s take a moment to reflect on what’s behind us and envision what lies ahead.

A year ago, I wrote about the transformative impact of generative AI and compared it to historical, technological revolutions like the printing press, inspired by James Burke’s Connections.

2024 was indeed a year of building. We saw creators, innovators, and dreamers harnessing AI tools to construct what once seemed impossible. Yet, with every opportunity came challenges, and with every breakthrough, more than a few MISTAKES.

Before I share my projections, though, I want to revisit the lessons we’ve learned and share my hopes for the next chapter of our journey.

Share

Reflecting on 2024: The Year of Building

Last year, we embraced the power of creation like never before. I know I did! Generative AI tools gave rise to a wave of innovation: entire applications coded in days, AI-assisted art that left us speechless, and models capable of human-like conversations that many cannot discern as machine or human.

One of my highlights was seeing more people adopt tools like Pinokio for experimenting locally and Perplexity.ai for intuitive research, to name a few.

Many more people still have yet to discover the value of these tools, but it’s great to know that it’s been helpful and that I’m helping readers like you find the most impactful opportunities!

But let’s not ignore the bumps along the road. AI MISTAKES reminded us of the importance of attention and humility.

AI MISTAKES: Lessons Learned in 2024

Here’s a quick top 10 of what I saw as the biggest AI MISTAKES of 2024:

1. Google Gemini image model errors (they shut it down)

2. Google AI Overview giving bad answers from Reddit.

3. Twitter (X) AI based “Explore” feature summarizing tops posts

4. Academia trusting plagiarism detectors as AI detectors (and punishing innocent students because of it)

5. Article generation with bad facts (toxic mushrooms, sports magazines and more!)

6. Microsoft’s Recall (which was such a privacy blunder they had to recall it)

7. AI models faking alignment

8. Using AI for scams (from bots, to phone calls, to images; CEOs and more)

9. Misconfiguration of AI tools like copilot, exposure data that should never be accessible to the whole organization

10. Job application interviews using an AI to ask the questions.

These are in no particular order, but it’s safe to say that while the innovations keep coming, not all AI releases and deployments in 2024 were good. Given the pace of innovation, I’m confident that 2025 will offer ample opportunities to learn from others. However, let’s also not be afraid to make MISTAKES and explore new ways to use AI to be a force for good in the world.

Also, I’ve built a new ontology to evaluate the many AI MISTAKES like these, so look for new and improved AI MISTAKES case studies in 2025.

These lessons underscore the need for responsible experimentation, ethical foresight, and continuous learning. I believe it’s possible to make great progress, learn valuable lessons, and do so responsibly and appropriately.

So, What’s Ahead in 2025?

If 2024 was about building, 2025 will be about agents. “AI Agents” will redefine productivity, tackling everything from task automation to strategic problem-solving. Imagine tools that don’t just respond to you but actively anticipate your needs, executing tasks as your digital collaborators.

I’ve already been working with these tools in 2024, but we’re far away from peak AI Agent territory. I think we’re really just getting started. This feels a bit like playing with GPT 3.0 at this point. While there are some places where it’s a mature experience (using Perplexity.ai’s Pro capabilities is definitely like having a research assistant), other features like Anthropic Claude’s PC control still have much work to be done before it’s wise to trust it to complete tasks without accidentally running batch command to format your HDD.

I believe we are quickly approaching a point where it will be easy to tell which organizations have a clear vision and direction for their goals and which are just following others' lead.

If you haven’t yet discovered your vision, maybe you need some help. You might need to use the MISTAKES Framework or something even more detailed (like the Learn from MISTAKES with AI course, launching a fresh update in 2025).

“AI Agents” aren’t the only development we will see in 2025. While they will most likely get the most attention, I’m afraid to say that the problem with phone call spam is only going to get worse.

In 2024 we saw multiple solutions around AI models being able to conduct phone calls. This can be good for you if you are the one making the calls. I can’t promise this is going to be good for anyone on the receiving end, but there are other solutions that you can deploy for taking calls, they even use AI.

Self-driving cars and humaniform robots are another area that is similar to the AI Agent developments but also decidedly different. Later this year, more than one company is expected to receive regulatory approval in multiple US markets for transportation. Jensen Huang stood on stage with many robots last year, announcing project GR00T, and I think we will see at least one company deploying robots for customer use.

Speaking of reaching reliable capability, I also think we will see significant adoption of Generative AI in large organizations. Many software and solutions providers are now able to deploy generative AI to specific use cases in a more turnkey way. It won’t be everywhere, and of course, there will be places where it reduces productivity instead of increasing it. Still, we will see significant contributions to ROI at more than an individual contributor level. The key question is: which users inside large organizations already utilize these capabilities but do so in the traditional 'shadow IT' way?

The last area I want to call out is how we use these models will change.

Looking ahead, I believe we’ll see monumental shifts in how AI integrates into our lives. Here are my predictions:

1. Smaller Models, Bigger Impact: Localized AI will become more accessible, empowering individuals and small organizations.

2. Generative Robotics: AI-driven automation will move beyond the digital realm into physical spaces, from manufacturing to personal assistance.

3. The Rise of Intelligent Systems: Tools like AI-powered wearables, interactive devices, and hyper-personalized assistants will make AI ubiquitous and easily accessible beyond the PC and phone.

4. Action Enabled Systems: Multiple integration capabilities will collide to bring an inflection point to how we use the models to take action across integrated systems. (Eric Schmidt calls it “text-to-action”)

Beyond these four key factors, some well-known founders are jumping back in and getting directly involved in developing AI technology to ensure the longevity of their companies.

These founders have returned to working directly with the companies they were previously CEOs to drive AI initiatives.

Jeff Bezos (Amazon)

Eric Schmidt, Larry Page and Sergey Brin (Google)

Bill Gates (Microsoft)

Michael Dell (Dell Technologies)

Getting such direct attention tells me that 2025 is going to be another big year, and the news and releases will keep coming.

The entertainment industry is sure to see more rapid changes. In 2024, we saw the first fully generative AI video commercial with Coca-Cola, and I expect we will see a 25-minute program or longer) in 2025. The progress we’ve already made puts this milestone well within reach.

Lastly, I anticipate at least one breakthrough startup (with under 100 employees) achieving significant success and profitability. So many start-ups are working on these use cases that I believe at least one is statistically guaranteed to make a splash. I don’t expect it to be OpenAI, but it’s most likely to be a B2B implementation where the profits are not directly tied to vast computing power and utilization requirements.

I’ll admit that there’s no big stretch I’m taking on these predictions, but it’s also true that even without wild predictions, we could see significant and impactful changes even if just these few developments materialize.

What’s your forecast for 2025?

Leave a comment

Build, Learn, and Experiment

I’m currently evaluating several agentic tools for coding and specific use cases. I haven’t completed my work assessing these, but since these are getting such good commentary and attention, you might like to know what I’m investigating. Who knows, maybe we can learn faster together as a community, so let me know if any of these grab your attention and keep it!

• GitHub Copilot (AI coding): https://github.com/features/copilot

• Replit (AI coding): https://replit.com

• Windsurf (AI coding) : https://codeium.com/windsurf

• Vercel v0 (AI coding): https://v0.dev

• Bolt.new (AI coding): https://bolt.new

• Cursor.ai (AI coding): https://cursor.ai

• Relevance AI (agents): https://relevanceai.com

• Abacus.AI (ai assistants): https://abacus.ai

• Make HQ: (workflows)L https://make.com

• ElevenLabs (voice agents): https://elevenlabs.io

Closing Thoughts

As we close the chapter on 2024 and embrace 2025, let’s remember that MISTAKES are stepping stones to growth.

Whether you’re a seasoned expert or a curious beginner, what’s ahead is your opportunity to create, innovate, and, yes, make MISTAKES.

This leads me to my final and most important point.

Let’s build together! Are you considering a particular use case? Do you have specific questions? Are you trying to solve a specific problem with AI but don’t know where to start? Let me hear from you. Call me, schedule an appointment, reply to this email (I love it when you read and reply!) or comment in the Substack app.

Subscribe now

I’m also looking into having guests on my YouTube channel or starting a podcast. It’s fantastic what discoveries can come through great conversations, so please let me know if you are even just a little bit intrigued by this idea.

Let’s push boundaries. And most importantly, let’s keep learning.

As always, I appreciate you reading the newsletter, subscribing to the YouTube channel, and joining me at the next AI MISTAKES Community Event. Here’s to a year of growth, discovery, and transformation.

Happy New Year!
Daniel Lemire

Share

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

My Thoughts

I find it fascinating how AI is transforming our work and creativity. Seeing how a simple prompt can lead to rapid and insightful responses is incredible, allowing me to extend ideas easily. However, not everyone finds AI so enchanting; we'll explore that further in this issue.

It’s still magical to type something in and receive a nearly instantaneous response that answers my questions or brings new discoveries right to my screen. Even more valuable is that I can keep going, keep iterating, and keep investigating and growing my knowledge at an incredible pace.

Shakespeare's sentiment was right. It would be wrong to forsake what we have been so blessed to be given. But not everyone wants to use AI because it helps learning.

For this reason, we should seek to make man better, not replace him. We should strive to improve our faculties rather than allow them to fade away with lack of use.

We have myriad examples of what happens when we disengage from the thought process and let the machine do all the work without proper human attention (here’s my favorite if you’re interested).

So, I encourage you not to allow AI to remove your skills, whatever they may be, but to use AI to sharpen them.

• Write better prose

• Create better software

• Speak to your customers

• Convey your value more clearly

This is only the beginning of how AI can help us learn and improve. The problem is, what if someone is just using AI to do all the work? It’s already impossible to discern what is written by a large language model like GPT-4 from what is written by a human. Worse yet, we often think that human writing is better than AI until we discover that AI wrote it; then, it’s not as good as a human can write.

In a recently published paper, GPT-4 is judged more human than humans in displaced and inverted Turing tests. This shows that humans and AI aren’t particularly good at discerning between AI and humans. Humans don’t even get a 50/50 chance; they’re just wrong most of the time. Even the AI does a terrible job of discernment.

So, instead of trying to compete against each other, let’s make AI the foundation of improvement. Make AI MISTAKES, so you can improve and understand what’s possible. That way, when the time comes, you’ll be better, as will your choices about when to use AI.

Leave a comment

Notable Headlines

• SearchGPT: A limited release of their new search experience.

• Llama 3.1: Open models are making great strides. What’s new? Improved multilingual support, longer context length, and better reasoning abilities

• Google Research gets silver in math: Google's DeepMind AI systems, AlphaProof and AlphaGeometry 2, achieved a silver medal level in the 2024 International Mathematical Olympiad

• Grok-2 makes big gains: Elon Musk's xAI has significantly improved Grok-2 and Grok-2 mini's performance. Grok-2 now ranks on the Lmsys Chatbot Arena leaderboard

• Apple’s model is open!?: Apple has released several open-source large language models (LLMs) called OpenELM, designed to run on devices rather than through cloud servers. These models, along with complete training frameworks and evaluation tools, are now available on the Hugging Face Hub.

• GPT-4o Mini changes the pricing game: OpenAI has introduced GPT-4o mini, a more affordable version of its powerful GPT-4o model. At 15 cents per million input tokens and 60 cents per million output tokens, it's 60% cheaper than GPT-3.5 Turbo.

• FLUX makes a splash: Flux.1, developed by Black Forest Labs, is a new AI image model that challenges industry leaders like Midjourney. With 12 billion parameters, it surpasses competitors in image quality and performance.

Learning, Tools, and Experiments

Fine-Tuning your Model (for free with OpenAI for the next month)!

OpenAI has announced free fine-tuning for GPT-4o mini through September 23, 2024, allowing developers to customize AI models for their specific use cases.

Fine-tuning is particularly effective for tailoring AI models, but I can already hear you asking, “Why!?” Ok, sure, so here are some common use cases:

Let’s say you want to create an AI model that can write just like you. You can make a model that sounds like you in style, tone, and vocabulary. But there’s plenty more. A fine-tuned model can help you improve the reliability of your desired outputs. This allows for correcting failures to follow complex instructions or niche-specific situations that don’t work well with the large models alone.

The process involves preparing a dataset of example conversations, uploading it to OpenAI's API, and creating a fine-tuning job. You can use many sites and videos to learn how to do it, and I’m hoping to show off how I do it in a video, so watch the channel for more details!

You may not want to try this, but now you know it’s possible—which also means anyone can do it! While this free option is available now, it might be the best time to explore it. Regardless, you can bet that there will be more opportunities in the future as the costs continue to decline and the technology improves.

GPT-4o mini is available for free fine-tuning until September 23, 2024, with each organization receiving 2M tokens per 24-hour period at no cost. This initiative allows developers to explore and leverage advanced AI capabilities without immediate financial constraints.

Share AI MISTAKES

AI MISTAKES

AI Detection is incredibly difficult. This month, let’s be less concerned about whether or not humans produced work and more about how we’ve learned and improved from the process.

Recent research indicates that over 80% of workers use AI without telling their employers. In the classroom, where students must apply their intellect to their assignments, it’s not much of a stretch to imagine that they, too, are using AI to get their work done without anyone else knowing.

In the classroom, though, it’s about more than plagiarism. It’s about ensuring students get value from a hard-earned education. The question is, did they learn anything in the process? If no learning took place, was the exercise even worth it? I don’t know about you, but who wants to grade papers just to grade papers? Not me. As a grad student, I was a TA, and it’s not a fun activity.

For learning to be truly valuable, struggle must be at least part of the process. It’s not about roll calls, grading papers, or filling out scantrons (which I just learned ended last year). It’s about what students learn.

The problem is that shady software vendors out there tell you they can reliably detect AI writing, and it’s just not so. AI detection tools are often inaccurate and unreliable. They can produce both false positives (incorrectly flagging human-written text as AI-generated) and false negatives (failing to detect AI-generated content). This unreliability can lead to unfair accusations against innocent students or failure to identify actual instances of AI use. The real problem here is that relying on AI detectors can damage the trust between teachers and students. It creates an atmosphere of suspicion and surveillance, which can negatively impact the learning environment and student-teacher relationships.

The AI tech is rapidly evolving, so it’s also expected that while their detection capabilities will likely improve, they will struggle to keep up. This could lead to an endless cycle of trying to outsmart increasingly sophisticated AI systems, diverting focus from actual teaching and learning. It’s just not the direction we should go.

AI Dectors are more of a problem than a solution.

AI Scoring is Misleading

AI Detection Impacts more than Scores

AI Detection Doesn’t Solve Cheating

Use AI with Caution

AI Detection is a Problem for Teachers

Care needed when using AI Detectors

Using AI to Detect AI Writing, What Could Go Wrong?

So what to do instead?

Since AI detectors offer little meaningful educational value regarding the use of AI, educators should consider the following approaches:

Foster Open Dialogue by engaging students in discussions about AI use, its implications, and ethical considerations.

Create an environment where students feel comfortable discussing their use of AI tools. Below, I’ve provided a list of resources to help with this.

Please provide a clear set of guidelines and expectations with practical ways to use the technology to help them improve what they turn in for grading without eroding the value of the process. This will help students, faculty, and staff as the entire community work effectively with this technology going forward.

Resources

There are also many resources available to help you navigate this emerging situation. While I’ve already created some helpful content, these resources align with your mission and problems. Please use these to better guide your next steps so we can skip over the bad MISTAKES and get to the heart of learning faster.

1. Khan Academy's AI for Education Course:
   https://www.khanacademy.org/college-careers-more/ai-for-education

2. Ethan Mollick's Resources:
   https://interactive.wharton.upenn.edu/teaching-with-ai/

3. Code.org's AI 101 for Teachers:
   https://code.org/ai

4. Google's "Generative AI for Educators":
   https://grow.google/ai-for-educators/

5. Microsoft's "Empower Educators to Explore the Potential of Artificial Intelligence":
   https://learn.microsoft.com/en-us/training/courses/empower-educators-explore-potential-artificial-intelligence

6. Wharton Interactive's "Teaching with AI" Resources:
   https://interactive.wharton.upenn.edu/teaching-with-ai/

7. EdTechTeacher's "AI Essentials for Educators":
   https://edtechteacher.org/ai-course/

Practically every school district worldwide has to derive a policy on where to take things next, and in my research, I came across the Plano ISD page on AI (https://www.pisd.edu/ai). They’ve already collected some great resources as well.

As I close out this edition, let’s avoid the harmful AI MISTAKES, the disastrous, damaging mistakes that could leave long regrets in their wake. It’s essential to encourage healthy experimentation, growth, and curiosity, but do it in a way that improves the mind. Go, try new things, start getting your hands dirty, and discover how this technology can help us grow and improve.

I want to hear from you; I read all your messages, just hit reply, and it comes directly to me. I’m here to help—learning, teaching, consulting, and helping others to learn from MISTAKES with AI.

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

My Thoughts

Is faster always better?

While quick results are appealing, the real value lies in starting—without action, speed is meaningless.

Starting is the most essential skill that we all need. It doesn’t matter how fast you can go without starting because you aren’t moving.

When it comes to getting things done, action is vital.

Regarding AI, the speed of inferencing for large language models (LLMs) is emerging as a critical factor in creating transformative user experiences and shaping the competitive landscape.

As industry experts and data suggest, rapid response times and efficient processing are becoming critical differentiators in the race to deliver more engaging and natural AI interactions.

This isn’t new information. Twenty years ago, amidst the Web 2.0 craze, we talked about how long a website’s visitor would be willing to wait for a response before giving up.

This hasn’t changed.

But when I talk about speed, it’s not just the user experience. It’s everything that comes along for the ride.

The exact details that taught us how long we should expect to wait on a web page also kept us returning to Google. Their results were fast and relevant and almost always led us to our desired answer.

Today, with the combination of cutting-edge telecommunications, computers, and AI, it’s faster than ever to find the answer.

When I built a prototype app with an LLM backend, I found that the processing delays from the best LLMs were not worth the wait. I was looking for real-time experience and willing to take a less detailed answer as a trade-off for speed. Granted, it was a particular use case, but now that I have seen what’s possible, I’m convinced this is one of the more critical factors.

And what I see in results from Google, Anthropic, and OpenAI is that all the major closed players agree. While they are all working to improve their models and results, all of them have worked to improve speed… because it matters.

Leave a comment

Notable Headlines

Claude Gets an Upgrade with 3.5 Sonnet

Anthropic has released Claude 3.5 Sonnet, an improved version of their AI assistant. All the effort here went into Sonnet, and it’s quite a bit faster than Opus, the previous king at Anthropic. The new model has always been faster than Opus, but this feels better at writing and has improved software engineering skills. I’ve been super impressed with the new artifact capability—so much so that last week's live video was nearly entirely dedicated to Claude’s new Artifact feature. It’s available for free via Claude.ai, so go try it out now if you have not already!

Google Flash: Optimized for Speed

Google has introduced Gemini Flash, a smaller and more efficient version of their flagship AI model that is build for speed. It also includes a long-context understand which is something new for a smaller and faster model. 1 Million tokens. (also, this week google has extended their 2 million token capability to all developers!)

Perplexity Launches Shareable "Pages" Feature

Perplexity AI has unveiled a new feature that enables taking a search and quickly sharing pages. I’m testing things out regularly and I've found this to be interesting and could perhaps lead to a whole new set of opportunities.

Are you interested in more content from me that is augmented with AI, something like perplexity pages where I choose the topics but let AI do the heavy lifting for the writing?

AI Video Generation Tools Proliferate

Video generation is garnering lots of attention lately. Kling (Kling is a mobile-only app, so you'll need to use your smartphone or tablet to access and use the service), finding the app on the relevant mobile stores), Pika, RunwayML and Leonardi.ai have all announced new updates and are really making things interesting as we all are still waiting on OpenAI to make Sora more available. This explosion of video generation is only the beginning.

Learning Tools and Experiments

I’ve recently had a few people reach out to me asking about opportunities to learn AI and how to best prepare for the future of their careers by learning about AI.

Learning

My advice might seem a bit counterintuitive to some. Still, it is backed by my personal experience and the historical happenings in AI and how it has evolved since the early days of computing in the 1930s.

Don’t make AI your ‘thing’; make your ‘thing’ with AI.

AI is only called AI until we find a practical way to implement it to a valuable capability. So, please take what you have that is already valuable and find the best ways to use AI to make it better.

Learn about AI, but do it in a way that is immediately value-generating by actually applying it. Learning is fun and fabulous, but it’s not creating value without application.

The section is aptly named because you need learning, tools, and experiments.

So, about the learning, here’s a simple exercise you can use to learn.

1. Choose an LLM app (Claude 3.5 or OpenAI 4o) and ask it to explain everything about something you are an expert at.

2. Identify the MISTAKES

3. Write about what you discovered.

4. Share what you wrote and tell people where these models got it wrong based on your experience.

Did you pick up some learning from today’s newsletter? Help me out and share it with a friend.

Share AI MISTAKES

Tools

In last month’s edition, I wrote that the doors have been thrown open wide. You don’t even have to pay to use the best tools. So what are you waiting for? Go and use them!

claude.ai

chat.openai.com

labs.perplexity.ai

developers.google.com

And just a few days ago, Google opened up 2 million tokens for Gemini for all developers. You’ll need to create a developer account, but there are many capabilities that you can test out totally for free. Google gives free access for new accounts for the first 90 days, so there’s no reason not to get started building if you have an idea you are ready to try out.

Use them all. Run the same prompt across all of them and compare. Do something you would already do (like with a Google search) and try it across multiple tools.

The tools are only the beginning, you have to get to the experiments to really lock-in your learning.

Subscribe now

Experiments

I‘ve been animating using Anthropic Claude’s new Sonnet 3.5 nonstop. Just make sure you go and enable the Feature Preview for “Artifacts” option in your profile settings screen.

This is definitely worth a closer look. Don’t miss out on the chance to learn and create something cool.

Speaking of creating valuable things, it’s already clear that we will use AI to do many things, and agents will be part of our future.

It’s hard to know exactly how this will work out, but if you want to start working with agents in a small way (which is the best way to start), you might like to look into a new GitHub repo. It’s called micro-agent, and if you write web code, you should definitely take the time to check it out.

AI MISTAKES

Things don’t always go to plan.

Sometimes, even when you have the right idea, you’re ability to do it well or with enough care is a challenge.

That’s where we find ourselves again with one of the large tech companies.

Google’s had their turn, OpenAI too. Now, it seems that it’s Microsoft’s turn.

Microsoft has played the game and discovered that while moving fast with the idea is essential, but action and adapting fast is MOST essential. Having a good idea isn’t enough.

On May 20th, Microsoft announced new Windows 11 operating system features integrating AI capabilities into the core OS.

I’ve been an early adopter and advocate of some of their great new features, but not every new feature that touts AI is great for everyone.

After announcing the recall, Microsoft has decided to reverse how it delivers the experience to all users.

It is essential to respond quickly when conducting innovation work. I applaud Microsoft for responding to what the community and users have been saying since the feature was announced.

I think the ideas that Microsoft was after with recall are essentially good, but not unlike what we discussed about AI devices last month; implementing it well is no easy task.

Ultimately, the things we do on our computers will become increasingly integrated into our lives over time. The future of that integration could come through many ways, from external devices, auxiliary storage, physical augmentation and especially robots. So, it’s important to keep an eye on these developments. Not all companies or organizations are will to change their approach based on feedback, so the fact that a big tech company has adjusted their approach is to be celebrated. Our just is to not only contribute our imagination, but also think through the consequences of our actions.

We only have to observe the changes over the last decade in how people use smartphones to see how this works. When was the last time you used a phone number for someone you know? I know I haven’t.

But more than that, not everything is a pretty picture. Sometimes, even having a wonderful device like a smartphone can be a bad thing if we let it take over how we interact with the world around us. We have to be vigilant about not just what we CAN do with the tech, but what we WILL do with the tech. Always be present and mindful.

So, we will see the adoption of recall like features and abilities for AI, especially as they relate to the things we want to remember and further process for insights, it just need to continue to evolve.

And speaking of recall, the best way to learn is to go and make MISTAKES. Nothing teaches like experience, and nothing improves recall like realizing what we did last time wasn’t the best.

So let’s be sure to learn from our MISTAKES while also letting go of the parts we want to forget so we can move forward and keep heading fast toward the things we want.

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

My Thoughts

MISTAKES HAPPEN. They are learning opportunities.

Life is messy. We often don’t have the full context.

It’s not uncommon for us to glaze over details and ask for things we don’t fully understand.

Is this human existence? AI chat sessions? Our relationship with other people?

YES.

But there is a powerful tool that helps to correct this. When we make mistakes, if we can get the right feedback, it becomes a valuable opportunity for improvement.

So what does feedback look like today? How can we do better? And what does that have to do with the latest and greatest in AI models, tools and techniques?

Over the last month, we’ve seen some incredible progress with open models, a new country has taken the scoreboard by storm, and political organizations (governments and others) are taking it upon themselves to regulate. The best models in the world keep getting better and there seems to be new capabilities released on a regular basis.

Take this generated image from above. There is no place in the world that looks like that. It does remind me a bit of a place I visited last summer in Scotland, but it’s decidedly not real.

The lack of reality is the problem. Whether it’s the feedback, the guidelines or even the understanding of what’s going on, as a society we have to keep learning.

Mistakes are being made. Some are human. Some are machines, and some are just plain lack of attention or intention.

So, where do we go from here?

My personal position is that the box has been opened. You can’t put it back in. We have to deal with what we’ve created for better or for worse.

I'm actually a fan of giving this technology to the world and setting it loose. I’m actually an advocate of going one step further. The way I see it, it’s on all of us to find the GOOD ways we can use this technology and put it to use, if only so that it drowns out the bad.

I keep seeing people say, “Oh, we can't trust anything on the Internet now because of AI!”

I can’t be the only one who thinks this, but I’m certain that ship sailed many years ago. Anyone with a blog can say anything at any time.

It is the automation and scale that make all the difference now, and that’s available to everyone.

So, let’s build.

Let’s deploy.

Let’s create!

Let’s find ways to overwhelm the negative with the positive. This technology has already seen incredible availability, applicability, and democratization.

I'm actually writing this newsletter because AI helps me make it happen.

The ball is in our court. Instead of letting someone else decide how it gets used, we can activate it today and advocate for what’s good and right in the world like never before. The sheer power of artificial intelligence coupled with automation can be used to address real problems today, and I know that we can overwhelm the bad.

It's actually the ultimate expression of freedom.

Many people have more to say than they admit, but fear overcomes their actions.

What can you achieve if you set your mind to it? Anything!

And now the reasons you have for not starting are eliminated with the power of AI.

You can create. You can build your dreams. You only need to start and be willing to start, knowing there will be MISTAKES.

So, let’s use all this great stuff to make the world a better place. Let’s focus on the good things happening and remember that iteration and feedback are key to improvement.

Now, let’s go learn, make MISTAKES, and build a better tomorrow.

Notable Headlines

• The models keep coming. The models keep getting better.

  • OpenAI Release GPT-4 update (‘Omni)

  • Google Updates Gemini

  • Meta releases LLAMA 3

  • Mistral releases 8x22B

  • Microsoft will showcase Co-Pilot updates next week

  • China’s SenseNova scores better than GPT-4

• AI Gadgets aren’t getting great reviews, but it’s only a matter of time before there’s a hit (maybe Apple?). Science fiction has already pointed us in the right direction and provided inspiration, but implementing great tech is hard.

  • Humane’s AI Pin

  • Rabbit R1

  • Limitless pendant ( ~ Aug 2024)

Leave a comment

Learning, Tools, and Experiments

OpenAI

OpenAI ChatGPT offers more features to free users. If you haven’t created an account with OpenAI because you’ve been avoiding paying a monthly fee, now is the time to do so.

Watch OpenAI's Spring Update Event

The announcement this week also means that anyone can start using GPTs as listed in the store.

But wait, there’s more! So much more. Check out Friday’s Live video covering all the big news items in the release.

And, while it’s not the kind of learning we most like, Once again Sam Altman find’s himself with some unwanted attention. In this case, it’s the voice ‘Sky’ in ChatGPT. It’s a little too familiar for some tastes. To make matters worse, Sam made mention of “HER” and that hasn’t helped.

Will this be a MISTAKE? Only time will tell. Meanwhile, it may be an opportunity for for learning.

If you haven’t seen the movie Her, it may be a good time to catch yet another movie about AI.

I hear you asking already: So what are the essential AI movies?

Here’s my quick rundown of the essentials.

• Her

• The Matrix

• I, Robot

• Ex Machina

• 2001: A Space Odyssey

• Terminator

Hollywood tends to choose the most violent examples, while I prefer the more helpful variety, so here’s a few science fiction books that give a deeper perspective:

• Ender’s Game + Speaker for the Dead + Xenocide (Ender’s Game is actually my most favorite Sci-fi stories, so let me hear from you if you read and enjoy it)

• The Moon is a Harsh Mistress

• Asimov’s Short Story Robot Dreams (I would say Asimov was the first to imagine AI in his stories, but it’s goes beyond Robot Dreams)

• Asimov’s Robot book series

   Caves of Steel > The Naked Sun > The Robots of Dawn and Robots and Empire (which follows the Foundation series, also recommended)

There’s plenty more AI learning in books, but I’ll call it a day with just the science fiction for now.

ElevenLabs expands into Music

Here’s a preview of ElevenLabs Music. Generated music, no edits. I’ve been playing with ElevenLabs for a few months since a colleague shared an audio snippet with me that replicated his voice, but it wasn’t him. It’s incredible what’s possible, and ElevenLabs is doing some great work. This music is impressive, but there’s still work to be done.

Checkout ElevenLabs and get started for free!

Speaking of what’s possible, checkout this “Reid Hoffman meets his AI twin - Full” video on X. Reid uses Hour One, ElevenLabs and a custom GPT (OpenAI) to create an AI twin.

Personal-AI is an assistant that works using Apple Shortcuts. If you have an iPhone, you don’t need to wait for Apple to release a better version of Siri, you can use it with Llama 3 today (the bad news… it’s still requires an internet connection).

Google Cloud Experiments

Google released a huge amount of information at Google IO this week, and I can’t possible cover it all. The truth is, for much of what they released, the value is in getting your hands dirty and playing with code and learning hands on.

Your feedback is valuable and I’d love to be able to connect with my readers.

Don’t be shy, I’m just a normal guy, learning about AI.

MISTAKES with AI

It wouldn’t be an AI MISTAKES newsletter without discussing some element of the framework, getting started, or pointing out the places where things didn’t quite go as planned.

This month, I’d like to discuss the biggest risk I see with AI.

Many are confounded and, quite honestly, scared of AGI. But before AI can exceed our human capacities, we will have to survive when people assume the AI is better and don’t bother to check things out for themselves.

In September of 2023, a team of researchers released a working paper: Navigating the Jagged Technological Frontier: Feild Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality.

I have discussed this paper multiple times, and this likely will not be the last. Unfortunately, as a society, we will have to learn this lesson over and over again, so it bears repeating for this audience.

To quickly summarize, it covers several important dimensions and shows valuable insight into how AI changes our abilities (both good and bad).

Taken from this report is the below graphic.

The most important part of the chart is that little read portion at the bottom left corner.

This data point represents what happens when humans stop thinking and only rely on AI.

To take directly from the paper, there are three critical ways this happens:

• Overreliance on AI: When individuals depend entirely on AI, they may become less likely to critically evaluate the AI’s output. This can lead to a decrease in performance for tasks that are outside the AI’s capabilities.

• Blind Adoption: Users accepting AI-generated solutions without scrutiny. This behavior can result in bad results, especially for tasks that AI cannot handle.

• Unengaged Interaction: Professionals who do not engage deeply with AI and fail to question its outputs tend to adopt AI suggestions without verification, leading to potential errors in judgment or decision-making.

In summary, Too much Trust and a Lack of Validation.

While automation of a person's work with the aid of AI is really valuable, if we stop thinking, we will have a problem. We’ve already learned this lesson repeatedly with tech like autopilot capabilities in aviation.

Don’t make this mistake. Keep your guard up. Use a critical eye. It’s easy to let your guard down and let it slip through. But with the right attention and AI at our side, we can build tomorrow better than today.

As I close out this edition, let’s make the GROWTH AI MISTAKES, not the silly, lazy, disastrous mistakes we’ve discussed this month. Go, try new things, start getting your hands dirty, and discover how this technology can help us grow and improve.

Go make some AI MISTAKES.

Share AI MISTAKES

In this edition:

• Quick Poll on Model Usage

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• AI MISTAKES

Quick Poll

Ethan Mollick (Wharton School at the University of Pennsylvania) recently shared the following on social media:

In every group I speak to, from business executives to scientists, including a group of very accomplished people in Silicon Valley last night, much less than 20% of the crowd has even tried a GPT-4 class model.

Less than 5% has spent the required 10 hours to know how they tick.

I’d like to know how many of my readers have experienced the incredible power of today's most advanced Generative AI Models. Please don’t feel like you need to pretend one way or the other. We’re all friends here.

If you answered ‘no,’ I’d love to know if I can help change this.

I’d also like to know if you would be interested in helping turn your boss, friends, or co-workers from a ‘no’ to a ‘yes’.

Don’t be shy—you can reply to this email. I love reading your replies to this newsletter!

My Thoughts

Over the last several years, I've thought deeply about what it means to accelerate into the future. My most memorable encounter with Accelerate comes from my work in DevOps. It's an excellent book.

ACCELERATE The Science of Lean Software and DevOps: Building and Scaling High-Performing Technology Organizations by Nicole Forsgren, Ph.D., Jez Humble, and Jean Kim. If you're an IT practitioner, I would wholeheartedly recommend this book. It contains some really great information about best practices and organizational transformation.

I know what you’re thinking: “What does this have to do with AI?”

I'm a firm believer that books like this, thoughts contributed by the DevOps community, have ultimately led to the incredible pace of change that we're experiencing in generative AI right now.

The evolution of the automation process and systems needed to build these cutting-edge AI capabilities are possible because of the practices that have been rigorously improved through building cross-functional teams that enable flow, faster feedback, and continuous learning

AI improvements and breakthroughs happen every day. Whatever you do, don't go on vacation; you'll get left behind because the pace of change is so rapid.

In all seriousness, though, I’ve settled on a monthly cadence because with all the announcements and developments coming at us, we must constantly take the time to zoom out and see what really matters. So that’s what I’ll continue to do here. I’ll share the news items that I think are relevant and share what I think is important.

Without a doubt, we are accelerating into the future, and the future of AI is very bright. My excitement grows with every new use case and application release because it means that more people will benefit from the use of these cutting-edge tools. This will provide an abundance of opportunities for all of us to learn, make mistakes, and accelerate the success of ourselves, our organization, and our customers.

Share AI MISTAKES

Notable Headlines

• Anthropic’s release of Claude 3 has dramatically impacted the scene by outperforming GPT-4 in some cases. Many consider it to be at least as good as, if not better than, GPT -4. I’m using it as my default, but I have yet to cancel anything with OpenAI.

• NVIDIA launched the Blackwell Platform, adding new and improved processing capabilities, and with some compromises, enabling 25x better efficiency than prior solutions. The Blackwell architecture includes several new elements designed to accelerate various tasks, and I was particularly impressed with the new NVLINK capabilities that enable the network to not only relay information across GPUs but actually to contribute to the mix. They also have a drop-in version so existing Hopper cabinets can be upgraded to Blackwell.

• Stability AI CEO Resigned, among other Key Talents. Emad Mostaque, the CEO and founder of Stability AI, has stepped down from his role for wanting to pursue decentralized AI. Peter Diamandis interviewed him on YouTube a few days ago if you’re interested in what he’s thinking.

• Sakana AI’s New Foundation Models by Merging Open Models. Sakana AI has introduced Evolutionary Model Merge, which uses evolutionary techniques to efficiently discover the best ways to combine different open-source models with diverse capabilities to create new foundation models.

• Google's system, SAFE, evaluates factual accuracy in text, especially from large language models.

  

  It uses Google Search to verify claims and shows promise for reliable information. Check out the paper on Arxiv here. I suspect this will get much of my attention in the coming weeks.

• X.ai released Grok-1 as open-source last month, but this month, it was announced that Grok 1.5 is on the way and will be available on X ‘soon’. It excels in math and coding tasks, showcasing improved performance.

• OpenAI and Microsoft are reportedly collaborating on an ambitious project to build a supercomputer named "Stargate," with an estimated cost of up to $100 billion. This initiative is part of a broader five-phase plan, with Stargate representing the fifth phase, expected to launch as early as 2028

Did I make a MISTAKE by missing a news story you found pivotal? I’d love to hear from you!

Learning, Tools, and Experiments

Last week’s LIVE broadcast covered the evolving role of prompt engineering. In the video, I covered the DSPy framework for algorithmically optimizing LM prompts and weights. While many of my readers aren’t likely to need this level of prompt optimization, automation, or rigorous process separation, it is a project that bears watching.

What I didn’t know a few days when I did the live broadcast was another tool called gpt-prompt-engineer that enables prompt optimization/automation with much less effort than DSPy. The creator, Matt Shumer, made it easy to access and use with a one-click run in Google Colab.

Andreeson Horowitz rates another inclusion in my newsletter. This time, they’ve shared an excellent report 16 Changes to the Way Enterprises Are Building and Buying Generative AI. This report is valuable to me because I have spent many years inside a large Fortune 50 company, and seeing that perspective through this report is helping me to continue improving my holistic view of the Generative AI industry. This report also calls out that if large companies are looking to invest, small independents and startups can reap some of the benefits at their size and scale by being choosy about testing and experimenting with this technology at a size and risk envelope they can tolerate. If there’s a message I’d like for you to take away from this newsletter, it is that you don’t have to be an enterprise to reap incredible value from Generative AI. It’s time to get started experimenting if you haven’t already.

AI MISTAKES

The excitement around Generative AI is reaching new heights. Some are now calling it a bubble. I don’t think we’re in a bubble at this moment, but it’s always important to temper enthusiasm with a dose of reality, so here are some ways that AI MISTAKES have been made.

As always, deploying and using this technology is essential, but we must do so with an eye toward creating value and mitigating unnecessary risks that lead to sloppy mistakes.

First up this month is a sobering story about the use of AI to identify mushrooms. In 2022, after a significant rise in mushroom poisoning, Australian poison researchers tested out a set of applications that identify mushrooms using photos. The results were not impressive. The worst outcome was that some of the apps identified toxic mushrooms as edible. It’s essential to recognize that these tools are getting better all the time, but it may be best to choose a use case that doesn’t include accidental death by poisoning.

The second story I came across this month is a use case intended to help people better understand legal compliance issues in New York City, which may be providing information contrary to its stated intention. NYC announced a program in partnership with Microsoft to help businesses understand and comply with the law regarding buildings, tenants, and other relevant business items, such as employment. So far, it’s not going perfectly. The article calls out some examples of where things go wrong. Because the chatbots are new, this could also be a user interface and training issue. It’s possible to apply these generative AI systems to help people find reliable information quickly without sacrificing quality. We can do better.

Subscribe now

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• Using the MISTAKES Framework

• AI Mistakes

My Thoughts

Science fiction is an incredibly useful tool. When it comes to imagining what's possible, we can play with new ideas at a different level by creating an artificial representation of reality from which we can control, manipulate, and derive additional insights.

But the two major events that happened in AI this month are not so far apart. Google got called out for how it represented reality. Sora has people reacting with fear and excitement about the incredible representations it can create for moving pictures.

These reactions are to be expected, and we are far from done.

Regarding the Google Gemini controversy, I sincerely believe that we need a “bias knob” that lets the user dial in the bias the system uses to construct replies. We all try to limit our bias, but I think it’s unrealistic to expect “no bias”.

We’ve built mechanisms like the “Op-Ed” in journalism decades ago, and I think it’s time we acknowledge the complexity involved in how these systems respond to what we ask. We should try to eliminate bias where possible, but it’s not always possible, nor is it always appropriate. It’s complicated, but this is where I’m at. If you disagree, I read every reply, so tell me how you really feel!

The more we constrain the responses, the less capable and useful they will become. Controls are warranted, but if they are controls, you also know what you will get. We already have the tools with the models to adjust temperate (that’s easier because of the math involved in predictability), but if we are tweaking things, it needs to be clear. At a minimum, system prompts need to be published or disclosed in some way so that we understand the “unwritten rules” that govern the systems. I’d really rather it be an open process than a closed door. We learn faster when it’s open and new ideas can be contributed.

As for the Sora announcement, it’s clear that OpenAI understood that the reactions would likely need time to settle down and to be well received and understood by the public before the released it for use. It appears that they were quite intentional about releasing some details and representations of what it can do but not giving access to the tool itself. It’s also important to remember it’s just a moving picture. They look amazing, but movies require more effort than moving rendered images around a screen.

I've seen a lot of reactions to the Sora videos. If you were a regular viewer of the YouTube channel, you saw that I was stunned by how high fidelity, high quality, and consistent the outputs were.

But as I said in the live broadcast, we don't know what it took to produce those. But I can tell you that I have friends in the movie, TV, and film industry, and it’s already garnered an article about Tyler Perry putting a pause on studio hires.

Looking back on the December edition, I projected we would have a full-length movie created with AI. That seems even more likely now. And if anything, the release of Sora has just helped more people to understand what’s happening and how fast. ElevenLabs is already working on adding sounds and audio, and this week, Alibaba released a showcase of their new Emo, which actually uses the lady from the Sora video to show off audio and voice augmentation work.

AI is going to impact Hollywood, television, music and art productions. How long will it take for an industry-wide impact? That’s less clear to me because I’m on the outside, and headlines don’t do well at contextualizing entire industries. But have you ever sat in the theater and read the literal hundreds of names it takes to make a film now? It’s an absolute army of people for any work of sufficient length or complexity. Undoubtedly, at least a few of those job roles can be impacted by what AI can do today.

Should we all be worried now? No, but I do think it bears consideration. There is an opportunity here if you step back and consider the possibilities.

So, if you are worried about what AI can do in doing your work, don’t delay. Start working with these tools now so that you are the expert on the new tech and tools. This puts you in higher demand, and better yet, if you have the vision, you can be the one doing the hiring for the next big film.

If you are not in this place, or it’s not clicking into place, please read the section at the end. I have a great tool to share with you, and I’d love to connect with you and help get you on a better track. This is part of the AI MISTAKES mission.

Leave a comment

Notable Headlines

Google Launches new Gemini Chatbot capabilities, complete with incredible abilities to generate highly detailed faces, and then the CEO apologizes about what it generates after some changes are needed

If you want more details, watch last week’s Live broadcast, but more work is needed across all AI generation systems.

Nvidia's market capitalization topped $2 trillion before falling below the mark again.

I expect the inertia to continue, but other solutions are coming to market this year.

Jenseng Huang, CEO of NVIDIA, argued that AI's ability to understand and execute commands in human language could democratize programming, making everyone a programmer without the need to learn specific coding languages.

I’ve taken to calling prompt engineering “Natural Language Programming” because it’s the new form of programming. Computers will do more on their own, so he’s not wrong.

AI startup Groq went viral this week for its new chip that delivers near-instant chatbot response times.

In summary, really, really fast. Not cheap.

Google struck a $60 million content licensing deal with Reddit, allowing Google to train its AI models on Reddit content.

Not the last content licensing deal we will see this year.

Google open-sourced a family of AI models called Gemma.

Open models are vital. I’m not particularly excited about Gemma; it will improve, I am sure, but I am excited that Google has open-sourced it.

ChatGPT can now remember everything it has been trained on.

I’m still processing this one, but it’s an upgrade.

OpenAI unveils Sora, a text-to-video model that produce complex and much longer video

This will have an impact in more ways than one. I can’t wait to see what happens in open source as a response.

Learning, Tools, and Experiments

Microsoft has a great Generative AI for Beginners program on GitHub. You do have to sign up for access to OpenAI on Azure, and everything is in Microsoft’s way of doing things, but if you are already on Azure, it’s a great way to dig in.

If you want to go really deep, you might be interested in Andrej Karpathy’s Tokenizer Mini Byte Pair Encoding project on GitHub.

Stability AI has shared more on their upcoming Stable Cascade (text-to-image) for consumer-grade systems. You can play with the preview here.

If you have a nice desktop system and home and happen to have a fairly recent NVIDIA GTX card, you can play with their new Chat with RTX. If you want to do more with your own documents and you have 36GB of storage available, go check it out. You can also take a closer look by watching my recent live broadcast covering it.

And don’t forget to subscribe to the YouTube Channel. I do a live video every Friday, sharing the latest news, ideas, and great research discoveries. There’s more than enough to talk about every week.

Want more AI News?

AI MISTAKES

Before we get all serious, I do have to share some MISTAKES from this month.

So, no AI system is perfect; there is always room for improvement. And also, because they are not perfect, we have opportunities that are not to be missed.

This is where I want to take a moment to discuss mindset.

Having been through many circumstances, I now far better appreciate the value of looking at things in the proper light. I want to call out the absolute necessity for everyone regarding mindset and how it becomes an advantage for you.

Because if you look at the news about AI, and all you see is “I am going to lose my job”, you need to reset. Take the time and effort to find the help you need to get yourself in the right frame of mind.

This is so important that I’ve made it the first module of the premier master course with AI MISTAKES.

I created a framework as a guide to make it easy to remember and use.

If you're interested in the course, sign up for the waitlist to be the first to know when the next course becomes available.

The goal here is to put yourself in a position to take action. It helps drive the question: “How can I take advantage of the circumstances?” instead of being a victim or passive observer.

If you show up with the vision and know how to use the tools, AI can likely help you execute the vision. Then, you can be part of the story, looking for great people to help you execute.

We might all think we have the best ideas. And then see someone else do it and go, Hey, that was my idea. And my response now is, yeah, but you didn't do anything with it. In today’s world, it’s increasingly true that ideas are a dime a dozen. If you have an idea, share it. Build the vision, and people will start to show up. But more importantly, if you care about that idea, find an action to take to execute it.

Now that we’ve covered mindset, I want to address the section header: AI MISTAKES.

What I see, more and more frequently, is people making the mistake of underestimating what AI can do.

I hope you can see from the headlines that AI is becoming more capable of research and development efforts across hardware, software, algorithms, and data.

What are the mistakes that it's making today? AI MISTAKES aims to understand how to think about AI, use AI, and apply it to our daily lives.

Don't let the mistakes slow you down. Embrace them. Learn from them. Get started today.

Go make some AI MISTAKES.

Share AI MISTAKES

Subscribe now

In this edition:

• My thoughts

• I need a favor

• Learning, Tools, and Experiments

• AI MISTAKES

My Thoughts

In 2024, we’re already seeing companies share their new AI wares. AI at CES was the hot topic. Wall Street earnings calls continue to talk about AI constantly in their announcement, and even before the new year AI has been mentioned hundreds of times. But a recent study published by BCG indicates that most companies aren’t diving in right away.

We are seeing adoption happen. And even if the company isn’t driving it, individuals are. More importantly, we already have examples of how enterprises use Open Source LLMs. (this hasn’t even been an option for very long!)

I’m not sure exactly how much of the audience crosses over between the YouTube channel and this newsletter, but if you are reading this and aren’t subscribed, do me a favor and check it out. I’m making it a habit of weekly streaming, and last week’s live on “The Future of AI is Open Source” seems to have touched a nerve. I do believe the future is open, and there are many reasons why I think this is the case. In the early days of the internet, so much of the Cambrian explosion was based on the fact that anyone visiting the website could easily view and learn HTML, CSS, and JavaScript. We’re seeing the same thing happen as models are shared, blended and reshared. Huggingface has established themselves as the go-to place for all things open-source models, and they’ve earned it well. They now have deals with all the big names working on AI (hardware and software).

The real value of the Generative AI revolution is that it opens up the technology to people who would otherwise not have the opportunity to use it. Therein lies my reason for starting AI MISTAKES. I want to help you learn about this so more people can use it. The faster adoption grows, the faster we improve, learn, and find new ways to apply it.

Not only is AI opening doors to opportunity, it’s also acting a catalyst to enable other emerging technologies that we have been talking about for many years. Earlier today, ARK Invest released their Big Ideas 2024 presentation, and they said it quite well with an excellent graphic to show how AI is driving convergence:

https://research.ark-invest.com/hubfs/ARK-Invest_BigIdeas_TechnologicalConvergenceMatrix.pdf

AI is enabling more rapid adoption, greater use case application, and market expansion. When we consider that this technology can ride on the commerce rails that have already been built via the internet, mobile devices, and globalization, I don’t think that saying this is a revolution in the making is an understatement.

With all of that said, these projections reveal the huge potential that lies ahead, and we must work to turn it into a critical advantage.

Let’s make more headlines like this: 12 New Jobs For The Generative AI Era. There’s plenty of work to be done; let’s get to work!

I need a favor.

I have previously shared my vision for AI MISTAKES, and I’m always looking for ways to serve. I’m in the process of building out some additional courses and offers, and if you’re looking for something specific, I’d like to connect with you more directly.

Over the next few weeks, I’ll be trying out some new offers, and I’d love to get your feedback. If you’re looking for education programs, from workshops to courses, I’d like to know more about your needs. Just reply to this newsletter and let me know what you’re thinking about or looking for!

In February, I’ll be launching a new subscription service as well, and I’m looking for willing participants to kick off the first month. If you’re interested, reply to this newsletter. You could win a FREE month (supply is limited)!

Learning Tools and Experiments

Cost Optimization

Did I mention that I am excited about the future of open-source models? Well, I am, and we’re going to see some great things happen this year.

One of those is a dramatic scale out of hybrid models. Organizations want high-quality output, but if you can pay nothing or less per query, why wouldn’t you do that.

So far, we’ve seen great success by the likes of OpenAI in making their general purpose models available. And they’ve even aggressively lowered the costs of using the calls. Last week, OpenAI introduced new options and lower pricing yet again.

So here is a solution that can help deliver improved costs, but helping make decisions about query orchestration.

https://github.com/leeroo-ai/leeroo_orchestrator

Architecture

Last year I shared a wonderful article provided by A16z (Mark Andreeson’s VC firm) and they’ve continued their work and have more to share. They’ve now launched this repo to help provide some great starting points for the key elements in architecture. If you’re building AI, this is worth your time or at least a bookmark.

https://github.com/a16z-infra/llm-app-stack?tab=readme-ov-file

Creation

Not to be outdone, there’s been a great deal of activity in solutions that everyone can use to save time and get their work done.

I use AI tools every single day. My favorite tools are:

1.) perplexity.ai
2.) ChatGPT GPT’s
3.) Microsoft Co-pilot (including windows 11 side panel)
4.) Pinokio (your own personal AI experiments lab)

I’m sharing weekly videos on my YouTube channel, so don’t miss out on this hour of fun and creativity every week as I go in-depth on what I am excited about and how I am using these tools to get more done in less time while also having fun!

AI MISTAKES

Magazine Publisher Arena Group’s Use of AI Content

“Ortiz isn't the only AI-generated author published by Sports Illustrated, according to a person involved with the creation of the content who asked to be kept anonymous to protect them from professional repercussions.”

None of the articles credited to Ortiz or the other names contained any disclosure about the use of AI or that the writer wasn't real, though they did eventually gain a disclaimer explaining that the content was "created by a 3rd party," and that the "Sports Illustrated editorial staff are not involved in the creation of this content."

AI is constantly improving. We probably already read more AI-written content than we realize. Watch for this mistake to be repeated in the future. This is a mistake on several levels. Firstly, almost all publications want to enforce copyright, but judgments have already been made that require attachment to a human for intellectual property. Secondly, they were pushing the boundaries of content creation, got caught, and then deleted the stories. Thirdly, the AI-generated content was never edited. The article references a story that listed “five simple…steps” but numbered them each “1.” a did that five times.

AI-generated content is here to stay, but don’t let it do the job for you. We’ve been down this road before. The MISTAKE that people make with automation is assuming that it’s perfect all the time. Humans aren’t perfect, and while we’d like to think that machines are, too, they aren’t. All processes need control, attention, and feedback. Nascent systems need this far more than mature systems. AI is great fun, but it’s more toward the nascent side.

Regardless of who was responsible for the quality of the content, Generative AI needs systems, processes, and procedures to eliminate errors and ensure consistency. Don’t make this AI MISTAKE.

Popular AI Chatbots Found to Give Error-Ridden Legal Answers
Large language models hallucinate at least 75% of the time when answering questions about a court’s core ruling, the researchers found. They tested more than 200,000 legal questions on OpenAI’s ChatGPT 3.5, Google’s PaLM 2, and Meta’s Llama 2—all general-purpose models not built for specific legal use.

I’m a big fan of the general-purpose models. We’ve seen multiple scenarios where GPT-4 (a general-purpose model by OpenAI) has better purpose-built models. The amount of knowledge and accurate responses that these systems can give is incredible! If you are building a new system and you lack data to build your own, setting up a trial of a general-purpose system is the best choice!

However, as we learn from this AI MISTAKE, not all scenarios lend themselves to general-purpose models. Many scenarios still benefit from independent models, augmented training, and proprietary information that is not available anywhere else.

2024 will be the year of open-source models, especially those coupled with other systems. The mixture-of-models architecture as demonstrated by Mistral MIXTRAL is very likely to gain significant traction and value in the near term.

Based on what we saw in 2023, using LLMs to identify case law and precedence is dangerous. We’ve seen this before. All seemed well when the use of Autopilot in airplanes was mandated over having a pilot-in-command to fly the plane manually. The problem, though, was when something out of the ordinary happened. Protocols required humans to take over control, but because of the mandated autopilot use, the pilot in command suffered from significant skill atrophy and either was unable to control the aircraft properly or lacked the decision-making practice to make the right call.

Nicholas Carr said it best in The Glass Cage: Automation and Us: How Our Computers Are Changing Us.

Whether it’s sophisticated aircraft or Artificial Intelligence tools, we tend to become complacent. Don’t make this AI MISTAKE. Build systems that ensure the right steps are taken when quality is paramount.

And in the meantime, use this everywhere with little or no risk.

Do some writing. Create some art. Learn about a new process or technique. There are hundreds of use cases for AI that you can use today at almost no risk.

And if you are having trouble finding one, please let me know. I can help.

Speaking of letting me know, if you read something here and have questions, reply to this newsletter. I read all your comments, and I love engaging with my readers.

I'm a huge fan of all of the great and incredible work done by James Burke on the Connections series.

I studied the Connections series in my graduate course also because one of the textbooks that we used was the Connections book by James Burke, it takes you through the arc and the evolution of technology.

We are in an incredibly unique time right now. I've taken the title of this month's newsletter from another video series with James Burke. The series was The Day the Universe Changed. I didn’t see it until I was in graduate school. The series is an even bolder take and walks us through the incredible changes across medicine, printing, science, industry, and more. AI would definitely be in the making if they were to decide it was time for a new episode after the added one for the internet.

The episode I saw on the printing press resonated with me deeply as a person of faith. Raised in the Lutheran Church, I was fascinated by how the printing press coincided with and amplified Martin Luther's Reformation. I had not thought about the technological implications and timing until I saw this presentation.

So, AI…

I want to evoke that sense with this moment in artificial intelligence. Over this year alone, we have seen some truly incredible days. There have been days over the last year where one story did not cover it.

And oddly enough, the pace is accelerating. We saw multiple days this year where you couldn't keep up with all the announcements and the news. I've dedicated a huge portion of my time to following the stories and understanding the concepts and implications of the changes we are going through with artificial intelligence right now.

Many analysts, academics, and others have been very focused on finding where things are headed so that we can help those around us navigate the incredible change that we are going through.

But the truth is, we are just getting started. 2023 was the year of the change beginning.

William Gibson said, “The future is already here. It's just not equally distributed.”

I believe this fully captures the current state of AI today.

The #1 Use Case in 2023

So many use cases are already available for generative artificial intelligence at this very moment. I have had the benefit of starting an entirely new thing this year. In large part, due to the changes coming to us through generative AI, I can create like never before.

Earlier this year, I undertook to create a brand new YouTube channel, and AI was a part of that story because I use AI software to create and edit and make those videos far better than they could possibly be on my own effort without those tools.

And this is the story across many different areas, but they are all about creating.

Editing is not a strong suit of mine, be it video or writing, but AI tools have made it possible to produce far more content than I could have imagined.

When I decided to start this newsletter, honestly, that was intimidating to me because any publication that I've ever created has been done with a team. And I'm now to create a newsletter entirely on my own. It's written only by me and edited with the help of AI (transcription and ideation using AI tools is also an amazing help).

As we step into 2024, we look upon the current capabilities where AI can assist with virtually any creative endeavor that you would like to undertake, whether it's writing, whether it's photography or still art, video creation and editing, music creation, and so on. Editing, music creation, editing and transformation. The arts of poetry and prose are probably the biggest.

But what interested me in generative AI was that I could write a few words and create an image that could take your breath away. And that is an amazing development. It's far more amazing than we give it the right credit for.

If you have ever considered being a creator, I honestly believe there is no better time!

Before we leave 2023…

I want to leave you with 3 important tips in 2023 for getting your hands dirty using AI.

1.) If you want to keep up with the latest AI, download and install Pinokio or at least look at what they offer. It’s completely free, and incredibly fun to click a button and play with the latest research in AI.

2.) Start using Perplexity.ai instead of Google for research. Trust me, this is better. (if you’re still unsure, check out my latest YouTube Video showing how I use it)

3.) Have a spoken conversation with ChatGPT. It’s also completely free. Just download the app (iPhone or Android) and use the headphone button to talk to it. You can do this alone, but it takes the chat experience to a new level.

BONUS: The future is already here with open LLMs. If you are a builder, start playing with all the new models using LMStudio. GPT4All still has some unique RAG capabilities, but the speed of access to the new models using LMSTudio is a game-changer.

Looking and Thinking Ahead

Now, there are two ways to see the changes ahead:

• It’s going to be great

• It’s going to be terrible

The future is what we make it. Don’t let someone else tell you the future; create it yourself, and you have control.

And this is where mindset comes in and why the first portion of my online course is about mindset. Because it's all about how you see the world as we step into the next revolution that is headed our way.

I cannot help but draw parallels to the incredible changes over time. Whether you'd like to choose the computer revolution, the information revolution, the industrial revolution, the emergence of machines, or something as far back as the printing press and the dissemination and creation of information like never before.

I think the AI revolution has elements of all of these.

We're looking at 10 years for the full impact to be felt, yet we are at the beginning of this revolution.

One of the parallels that has often been discussed is the evolution of Netscape in 1997 in terms of browsing and searching the Internet and how that became the window to the great World Wide Web.

It's not clear what that window is today, but I think at this moment, the closest we can get is ChatGPT, as it was the one that showed us what was possible. This is a tiny slice. We are still beginning to discover what's possible more and in new ways every day. New and emerging properties are discovered daily.

Looking towards 2024, I'm filled with immense excitement. The future holds opportunities previously unimaginable, marking a significant shift from gloom to groundbreaking prospects. Though this moment might be forgotten in a decade, its impact will shape our future.

As a technology professional over the last decade, work in information technology has been about building teams that can build software and systems.

As I look at what AI can produce right now as a product owner or product manager, having had some specific training and certification about owning products, I'm now imagining a future where I am the product owner and the developer. I am the founder.

It’s no longer out of the question to imagine myself as a SaaS product owner because I know what needs to be built. I can use artificial intelligence to write the source code, run the application, monitor the application, and do all of those incredible DevOps things I've been studying, sharing, and teaching others about for the last decade.

The toolbox has more tools in it than ever before. And instead of finding ourselves in a situation where it is too much for any one person to do, this arc of change with artificial intelligence means that one man can do it.

That doesn't mean I will only be a one-man operation forever, but for now, it means I have some incredible opportunities, and I have to think deeply about what the future might look like.

2024

So, with that in mind, let's look ahead to 2024 and talk about what's possible and what's coming in 2024. My prognostications for the year 2024.

2024 will be the year of building.

We will see far more opportunities for companies to be started to go from nothing to a

revenue-generating product in record time. They may not have billions or millions of users, but we will discover product niches that we could never have imagined. Just as we did in the World Wide Web's early days, job descriptions that we couldn't imagine would be created in the blink of an eye.

In 2024

• An entire movie created and produced (likely not in the theaters, but on a screen near you)

• 3D Object creation becomes mainstream (and works with 3d printers)

• Systems that run systems intended for human control will be completely managed by an AI system that uses a computer instead of a human.

• Unique models will also be the order of the day for solving real problems that general models don’t handle well.

• We'll see an explosion of open-source capabilities.

• Large language models will become ubiquitous across all device types

• You’ll be able to wear an AI watch

• Write with an AI pen

• Use AI headphones (that capture everything audible around you)

• Environmental monitoring accelerates with AI intelligence to predict ways to be more efficient

• You'll be able to create your own repository of information about your data.

• And we will see at least one person create a full-on, high definition, fully informed data set about themselves that you can interact with.

And it will be stunningly good.

The bad news is, if you thought there was disinformation in the past, it’s not likely to improve in the near term.

As we move into election season, past news of election manipulation, ad campaigns, and disinformation will be something entirely different than it has ever been before.

Ad campaigns and disinformation will be something entirely different than they have ever been before. Finding trustworthy sources of video, audio, or even text documentation in 2024 will become difficult and will bring an urgency to solving the authentication question.

Understanding what is authentic and what is synthetic will become a bigger problem, but this will become a priority we work toward solving. Blockchain and cryptography will nab the spotlight to help solve this.

AI MISTAKES IN 2024

In 2024, we will be able to build more systems faster.

And we'll quickly discover that our ability to create leads to more questions about what we should create and why.

And it is for this very reason that I want to share with you my plans for AI mistakes in 2024. In 2024, I intend to grow and expand community engagement, the number of courses I'm offering, and the kinds of available coaching.

I want to work and collaborate with you.

More events. More courses. More community. I’m also going full speed ahead on consulting and coaching. If this is at all interesting to you, let’s talk!

I want to help you understand your ideas so that the time you spend developing those ideas is maximally beneficial to you and those that it will impact.

Our ability to think deeply, dream big, and build anything will become more important.

Because you'll be able to build anything. And it's not about building anything. It's about building the right thing for you and those you serve.

As we close out the year 2023, I also need to share with you that part of my vision for 2024 is not only to provide unique, informative, interesting, and engaging content and education but also to share with you the stories that go along with the information that, as people, we have taken with us for thousands of years.

I want to put a renewed focus on the sacred information that has been handed down for centuries so that we can all understand the magnitude of that information and what it means for us as human beings.

I want to cover the things that you find interesting as well. So, if you have thoughts about what you'd like to learn, I'd love to hear from you.

I read every comment about this newsletter. Just hit the reply button. It comes directly to me. Let's have a conversation. If you're in the Dallas-Fort Worth metropolitan area, I'd love to have a chance to get together with you and explore what we might be able to work on together. I’m holding another in-person event on January 9th.

Whether that's an as-yet-undetermined podcast, building a Software as a Service application, creating a consultancy, or building a company that serves the needs of businesses and people worldwide.

There are so many opportunities for us to pursue today. I'd like to know what you're interested in.

I want to serve you and help you to solve the problems. It is through solving problems and helping each other that the future exists.

Because we can create like never before, solve problems like never before, and, in this moment, celebrate like never before.

Let’s make 2024 a year to look back up with awe, wonder, gratitude, and love.

Thanks, and Happy New Year!

For those of you who have subscribed to the newsletter, I sincerely thank you for signing up and reading these words that I share with you.

Thanks for being here and being a part of the story.

And here's to 2024 and the year ahead. Go make some incredible MISTAKES in 2024.

In this edition:

• My thoughts

• Notable headlines

• Learning, Tools, and Experiments

• Using the MISTAKES Framework

• AI Mistakes

My Thoughts

As the Generative AI revolution continues, a trend is beginning to form. Software companies are finding ways to integrate generative AI into their solutions regardless of industry.

It turns out that having built an AI model that can somewhat intelligently talk about everything can be integrated somewhere along the process of doing everything.

The biggest application of this technology is at a step in the process where MISTAKES can be made safely. This is what has (in large part) driven me to create the MISTAKES framework. AI is incredibly capable of helping us to understand how to start and how to make progress…. but it’s not really good at solving the problem completely, permanently, or even perhaps sustainably.

If you leverage a framework that allows for MISTAKES, you can obtain all the benefits of that intelligence and avoid the problems. No matter what technology can do for us, it should never put a human in a position to no longer ever understand the problem being solved.

The reality is that we’ve already been here. We know how this story ends. That book was written by Nicholas Carr in 2014. I can wholeheartedly recommend this book, especially today, because we already understand what happens when humans give up their thinking to a machine.

In the introduction to The Glass Cage: Automation and Us, Nicholas Carr shared an "Alert for Operators” published by the FAA on January 4, 2013. I won’t rehash the notice, but the essence of the notice was as follows:

The FAA learned through investigation, research, and incidents (some of which were fatal) that pilots had become too dependent on autopilot systems that reduced (or in many cases eliminated) the need for a pilot to control the aircraft directly.

What made the situation worse were the studies that showed that aircraft fuel efficiency was improved when autopilot was used over long distances. In other words, in a time when the price of fuel was having a major impact on operational costs, airline directives actually forced pilots to engage autopilot to save money.

I hope we don’t have to learn this lesson again, but that is where we are headed. So, my efforts to prevent this is to help you engage your brain and use this technology as a powerful assistant rather than simply handing over the controls.

Notable Headlines

DALL-E 3 Now Available for all Paid subscribers of ChatGPT

NVIDIA introduces TensorRT for Windows

An Open answer to GPT-4V: fuyu-8b

NVIDIA and Foxconn team up to build AI Factories

Learning, Tools, and Experiments

If you are into using local system-based LLMs, you might be interested in this repo full of full stuff related to LLaMA 2.

I’ve been playing with tools that accelerate my creator work. The two I have been using this month so far (besides ChatGPT and Perplexity) are Typset and Gamma.app.

Last but not least, I saw an interesting video for a new tool that connects with Figma called Builder.io. I haven’t even had a chance to play with this one, but it looks exciting.

As for ChatGPT, If you haven’t discovered the power of Custom Instructions, boy, are you in for a surprise, but let’s move into the framework to continue this discussion.

Using the MISTAKES Framework

For those of you who caught the most recent newsletter, you’ve had a chance to learn about the MISTAKES Framework. If you missed it, please go back and take a look. I’ll be referencing it here, but it has a more complete explainer. This is definitely my crowning achievement in 2023, and this section will become a regular part of the newsletter. The framework has an incredible amount of value, and I intend to help everyone understand how it can make a difference.
It turns out that AI makes the MISTAKES Framework even more valuable because it provides the needed structure for the building, evaluating, and using AI systems to create value.

One critical advantage of using a framework is the power it gives us to ask questions and get responses from Generative AI systems.

Eventually, I believe that advanced prompting skills will go away, and you will simply interact with these systems naturally and ask questions, but for now, it takes a bit more effort.

Here is a conversation I had with ChatGPT about using the Custom Instructions capabilities of ChatGPT. I gave it a very simple prompt:

explain how to use custom instructions in ChatGPT Plus

Before I entered this prompt, I filled out the custom instructions.

What would you like ChatGPT to know about you to provide better responses?

Our goal is to help people use technology to solve business problems.

We will use the MISTAKES Framework to answer queries; it is , a comprehensive working approach to innovation and problem-solving that allows us to work step-by-step.

1. Map: Identify the problem area, user needs, and context.

2. Investigate: Research and gather as much data as possible to fully understand the situation.

3. Sketch: Develop initial ideas, solutions, or prototypes.

4. Test: Validate the effectiveness of these prototypes.

5. Adapt: Refine and enhance your solution based on the feedback.

6. Kickstart: Implement or launch the final product or solution.

7. Evaluate: Measure the results, assess the effectiveness of the solution, and reflect on what could be improved.

8. Sustain: Maintain the solution, iterate on it based on continual feedback and assessment, and ensure it continues to provide value over time.

How would you like ChatGPT to respond?

Responses should be less than 500 words unless I ask for more.

Leep the language as simply as possible, and keep to an 8th-grade level in course content generation.

Responses should remain neutral.

The goal is always to simplify concepts and make things easy for others to understand.

For answers, we will use the structure of:

1. Explain how to use the framework step

2. What action do we take using that framework step

3. What is an example of using the framework item with the query.

4. What can I use AI to help me with for this step:

For any answer you give, also provide a suggested prompt that can be used to accelerate learning and success with the relevant MISTAKES Framework step.

As you can see from the length of this prompt, I got a lengthy response. And this is the amazing thing that happens when you apply a framework to generative AI. The answer is incredibly useful. It provides not only the way to start using it but also provides additional steps to take in adapting, using, extending, and sustaining the use of Custom Instructions. Your answer will vary because that’s how generative AI works. Still, I think you can see, without even trying it yourself, that there is tremendous value to be unlocked, not only by using Custom Instructions but by using powerful frameworks like MISTAKES!

1. **Map**:

- Identify the problem area: You want to use custom instructions in ChatGPT Plus to achieve a certain goal.

- Action: Understand the scope and capabilities of ChatGPT Plus and the purpose of using custom instructions.

- Example: Determine whether custom instructions can help in providing more directed or specific responses from ChatGPT Plus.

- AI assistance: Use AI to research the capabilities of ChatGPT Plus and to understand how custom instructions work.

Suggested prompt: What is the main goal of using custom instructions in ChatGPT Plus, and how do they enhance the interaction?

2. **Investigate**:

- Gather data: Look into the documentation of ChatGPT Plus, tutorials, or community discussions to learn about custom instructions.

- Action: Research the syntax, usage, and examples of custom instructions in ChatGPT Plus.

- Example: Find examples where custom instructions have been used to guide the model's responses in a specific direction.

- AI assistance: Utilize AI to help find and collate resources, tutorials, or forum discussions regarding custom instructions.

Suggested prompt: Can you find examples where custom instructions significantly improved the quality or relevance of the responses from ChatGPT Plus?

3. **Sketch**:

- Develop initial ideas: Create a basic outline or draft of how you want to use custom instructions.

- Action: Draft a few custom instructions based on your requirements.

- Example: Write a custom instruction to ensure that ChatGPT Plus provides concise answers.

- AI assistance: Use AI to help draft different variations of custom instructions to see which works best.

Suggested prompt: What are some different ways you can phrase your custom instructions to achieve the desired output?

4. **Test**:

- Validate effectiveness: Test your custom instructions with ChatGPT Plus to see how well they work.

- Action: Run several tests using different custom instructions to evaluate their effectiveness.

- Example: Test how well a custom instruction helps in getting more detailed answers from ChatGPT Plus.

- AI assistance: Use AI to automate the testing process by creating a script that sends various queries with custom instructions to ChatGPT Plus and evaluates the results.

Suggested prompt: How did the custom instructions affect the responses, and were there any unexpected outcomes?

5. **Adapt**:

- Refine your solution: Based on the testing results, modify your custom instructions to improve their effectiveness.

- Action: Make necessary adjustments to your custom instructions.

- Example: Modify a custom instruction to make it more clear or precise based on the testing feedback.

- AI assistance: Use AI to suggest modifications based on the analysis of testing results.

Suggested prompt: What improvements can be made to the custom instructions to better guide ChatGPT Plus towards the desired responses?

6. **Kickstart**:

- Implement your solution: Apply the refined custom instructions in your interactions with ChatGPT Plus.

- Action: Start using the refined custom instructions in your daily use of ChatGPT Plus.

- Example: Use a custom instruction to get ChatGPT Plus to provide step-by-step explanations.

- AI assistance: AI can help monitor and log the performance of the custom instructions over time, helping to identify any areas for further refinement.

Suggested prompt: How have the refined custom instructions impacted the quality of interaction with ChatGPT Plus?

7. **Evaluate**:

- Measure the results: Assess the effectiveness of the custom instructions in achieving the desired output from ChatGPT Plus.

- Action: Analyze the responses from ChatGPT Plus to determine if the custom instructions are working as intended.

- Example: Assess whether a custom instruction aimed at getting more concise answers is effective.

- AI assistance: Use AI to automate the evaluation process by analyzing the responses against a set of predefined criteria.

Suggested prompt: Are there any patterns in how ChatGPT Plus responds to the custom instructions, and how can these insights be used for further refinement?

8. **Sustain**:

- Maintain your solution: Continue to use and if necessary, refine your custom instructions based on ongoing interactions with ChatGPT Plus.

- Action: Keep a record of effective custom instructions and continue to refine them as needed.

- Example: Over time, refine a custom instruction to better suit evolving requirements.

- AI assistance: AI can help by continuously monitoring the effectiveness of custom instructions and suggesting refinements based on new insights or changes in ChatGPT Plus behavior.

Suggested prompt: How can you ensure that your custom instructions continue to provide value in your interactions with ChatGPT Plus over time?

If you enjoyed this section on using the framework, you’ll really love what’s happening inside my new course. Not only do I share more in-depth knowledge on how to use these tools, but I also share the methods that accelerate using them to get the actual work done as well.

AI Mistakes

I’ve started to schedule a weekly live session on YouTube, and I’m having fun doing it, so I’ll continue to do this for the foreseeable future. Please go check out the channel, and if you’re reading this immediately after I publish it, you can join the live session or catch the replay here:

https://www.youtube.com/live/heyFqZOghu4

Business Pro Online Cohort Course

Who is this for?

Are you a product owner, IT professional, developer, individual contributor, or analyst?

Would you like to be someone who not only knows how to use technology to but actually knows how to get things done and create value?

I will help you:

• Transform your perspective and approach to problem-solving

• Unlock your potential with practical AI skills and knowledge

• Extract real value from everyday opportunities

• Take action to make a difference for yourself and others

How it Works

1. Join a community of like-minded people

2. Learn from Daniel directly

3. Take action and make MISTAKES daily 

Check out the full course description at the link below and join the waitlist if interested.
Learn from MISTAKES with AI

If you have questions, have comments, or are just looking for some clarity, please don’t hesitate to reply to the newsletter, or feel free to grab a free consultation with me.

In this edition:

• AI for Value Creation

• The MISTAKES Framework

• Learning, Tools, and Experiments

• The Launch of Business Pro Online Course

Emerging Technology

I’ve been studying and applying emerging technology for over 25 years, and one thing remains constant. There’s always another new cool innovation that everyone’s talking about. It’s fun. I love it, but we also need to talk about the dark side of this illustration.

Today, you can’t open the newspaper without seeing some headline about AI. All the major technology companies have something new they want to tell us about every week that incorporates AI.

But does any of this shiny new tech matter?

Do I need an AI Copilot in Microsoft Windows?

Do I need AI in my sunglasses?

Should I be excited about getting a Generative AI answer in my Google search?

What we need to talk about is value creation. None of this cool tech matters if it doesn’t solve real problems.

AI Impact on Value Creation

Digging through the headlines and analysis, here are a few key points that have come across the headlines:

• AI can enhance workflows and decision-making to increase efficiency and revenue

• AI systems can be misaligned with company goals, lead to unintended consequences and outright misinformation

• Goal misalignment can cause inefficient use of resources or actions counter to human values and reverse productivity

• AI value creation depends on strategic alignment with business objectives

It’s this understanding that drives my mission with AI Mistakes:

To help people get the skills they need to be successful in the new AI-augmented world.

Imagine with me…

You're strolling through your property on a crisp morning, enjoying the fresh air and taking in the sights. 

Since this is your regular walk with your dog, you must have been on this path hundreds of times.

But today, something catches your eye…a glint of something metallic half-buried in the earth. 

A sense of anticipation builds as you kneel to investigate, wondering what this metal thing could be…until you begin digging it out and see a glimmer peaking through.

And it's a treasure that was hidden for over a century. 

This was the extraordinary experience of a California couple in 2013.

While on a morning walk, they unearthed the Saddle Ridge Hoard -

A hidden cache of 1,427 gold coins worth an incredible $10 million.

Maybe you're wondering why I'm telling you this treasure-hunting story.

Like this fortunate couple, many business professionals have a treasure trove waiting to be discovered. The difference is you don’t even have to search!

It’s unavoidable: Artificial intelligence has already changed everything.

William Gibson said it better than I can:

”The future is already here, it's just not evenly distributed.”

This is completely within your grasp… if you’ll only reach out and grab it!

It’s not even been a year since the release of ChatGPT, but you can already see a small group of AI enthusiasts (like me) surging ahead in their career growth:

• Using AI in everyday work makes them more efficient in their existing jobs and gives them new promotions and career-advanced opportunities.

• Launching AI-powered startups and small businesses, suddenly solving problems in ways that were impossible even 12 months ago.

• Setting themselves up as thought leaders for the AI-powered future – getting invited to teach, speak, and provide insights by organizations large and small. It’s every innovator’s dream.

There are ways to accelerate mastery and success with the right framework and tools.

Early AI adopters are already benefiting by making money and expanding their careers – and this industry has barely even begun.

I’ve built roadmaps. I’ve defined programs. I’ve helped people to discover that one critical insight that changes everything. The world today is changing by the minute because of the impacts created by the advancement of AI.

I want you to be successful, so I’m sharing what I’ve been working on for many months today.

The MISTAKES Framework

The framework is your map of the forest. You guide for the journey.

The secret is about getting the right things done at the right time.

It’s your express route to enable solving the right problem.

Combined with AI tools, it can move you quickly past roadblocks and get you to success.

Are you ready? Let's go!

M - Map

Begin by discerning the challenges, understanding user needs, and contextualizing the environment. A well-drawn map focuses your efforts on creating solutions that solve problems and generate lasting value.

I - Investigate

Leverage data to enrich your understanding and inform your approach. Thorough research becomes the bedrock upon which value-creating initiatives are built.

S - Sketch

Transform initial concepts into potential value drivers through ideation. You begin visualizing the interface, function, and architecture in the sketch phase, enabling you to see the future before you actually build it.

T - Test

Validate your ideas through rigorous testing to ensure they hold the potential to create value. This phase is essential for aligning your initiatives with your overarching goals for long-lasting success.

A - Adapt

Utilize test insights to refine your initiatives, focusing even more on value creation. Iterative improvement ensures you're on the most effective path to generating sustainable value.

K - Kickstart

Move to action, transforming your refined ideas into tangible solutions that create real-world value. It’s the pivotal moment where strategy becomes impact.

E - Evaluate

After implementation, measure the value created to gauge the success of your initiative. This evaluation provides the insights needed for ongoing improvements and long-term sustainability.

S - Sustain

The last phase is where value creation is cemented. Ensure your initiatives achieve and sustain their value through continuous assessment and iteration.

I launched the presentation of the reason and value of the framework live on YouTube, LinkedIn, and Twitter last Friday, and you can watch it here inside the newsletter now if you’d like to get the full experience.

I’ll do another workshop session on Friday at 4 pm US Central time. Come along and ask your questions, or feel free to reply to the newsletter and ask questions now, even if you cannot attend.

https://www.facebook.com/events/699077245430482

https://www.linkedin.com/events/7118268004408160256/comments/

Learning, Tools, and Experiments

Since I discovered Stable Diffusion in 2022, I’ve invested thousands of hours and dollars to expand my AI expertise and present, share, and discuss AI with people.

Here’s what I know:

• AI is for Everyone: Tools are everywhere, but you must know how and when to use them to achieve your goals and create value.

• Opportunity Abounds: It’s 1997, and you just connected to the Internet.
  There’s so much new in AI; if you could go back and tell yourself how big the WWW would be (and that it would be in your pocket with you wherever you go), you would do it. This is your chance to take an early lead and create compounding benefits by becoming an expert.

• Accelerate Your Visible Progress: AI not only helps to create things, but it can help you learn, build and deploy.

• No-Code Tools: You don’t have to be a coder.
  You can start building something with AI workflows today. If you can use social media, you can build with AI. It’s actually possible today to prototype your products or add AI to your daily workflow at work without spending weeks doing it. With the right tools and framework, you can launch a new product in weeks.

How long would it take to figure out even one of these independently?

How much trial and error would you have to endure?

How much money would you spend?

What else would you have to sacrifice to find success?

I’ve spent the last year experimenting, making mistakes, and learning. I’ve literally been using the MISTAKES Framework to find my way to success. Work with me, and we can Accelerate Your Mastery!

Today, I’m pleased to announce…

The Launch of Business Pro Online Course

What’s in the course?

• Nine Weeks of Comprehensive Learning: A new module is unlocked each week, allowing you to progress through our meticulously designed curriculum based on the MISTAKES Framework. Each session is recorded to work through it at your own pace. Every week, you get specific guidance on executing the framework and the AI tools and techniques that ensure you Learn from MISTAKES with AI.

• Weekly Group Consultations: An invaluable opportunity to have an interactive group session with me, where you can clarify doubts, discuss your project, and get professional insights in addition to the course curriculum.

• Exclusive Access: Membership in a private online community where you can interact with like-minded professionals and get additional support.

• Certification: Receive a certificate from AI Mistakes upon completion to validate your knowledge and skills in implementing AI without the common pitfalls.
  

To give you a small preview of what’s inside the course, here are some valuable tools from the first session of the course.

• The 4C’s of a successful mindset
  
  A handy 1-page poster that provides a useful way to remember the key elements that matter for your mindset.

• The Mindset Checklist
  
  A handy 1-page checklist to help you change and set your thoughts about embracing mistakes and creating success for yourself, along with specific prompts you can use with GPT-4 to work with the AI to learn and improve how you think about mindset.

I’m not going to lie: Like it or not, the AI revolution will happen with or without you, and I want to see you succeed and take advantage of this huge opportunity. If you’re not ready to start, book a FREE 1:1 session to discuss how I can help you move forward.

So, if you're ready to get started….

Take The Course

I hope you’ve found this month’s edition packed with value, but I always look for ways to improve. I love hearing from my newsletter readers; let me know what you’re thinking and how I can help you create value. Replies to this message come directly to me.

In this edition:

• My thoughts

• So What’s Next (Learn from MISTAKES using AI)

• Notable headlines

• History: Apple’s Siri, Amazon’s Alexa, and Microsoft’s Cortana

• Learning, Tools, and Experiments

My Thoughts

For me, AI has become an already indispensable companion for my daily work. I use it to create art, test ideas, brainstorm, improve my writing, research, and learn, not to mention how it has completely transformed how I write code. Each new day brings announcements and developments from new and established companies, sharing their unique solutions and latest advancements for us to enjoy.

Many ways we can use these wonderful tools are still evolving, but if my experience is any indication (and I really believe it is), having a service that already knows and understands us is in our future. I’ve been using the ChatGPT custom instructions, and since most of my work is around my current project (which I have described in detail using the customer instructions), ChatGPT already has good context when I create a new chat thread. It’s just a tiny thing, but it’s been terrific for my productivity. You can do this for yourself now since ChatGPT has also made custom instructions available to free users. This is just one of the many customizations I have seen with generative AI tools. Midjourney has custom styles you can define; Lenoard.ai will build a model using images you upload (for free), not to mention that app (Lensa) that blew up the App Store last year for people to customize their selfies.

As we dig deeper into what’s possible with LLMs, the tinkerers, makers, and hackers search for ways to push the limits. Everything from ‘superprompts’ to ‘jailbreaking’ to finding ways to ‘data break’ the possible responses and interactions with these generative AI systems.
Some are approaching the testing of LLMs and their response to prompts just like they would any Information Technology system by “red teaming.”

Red teaming is a practice in information security where a group of ethical hackers, known as the red team, test the effectiveness of its security measures. The primary goal of red teaming is to identify vulnerabilities, test assumptions, and reveal limitations and security risks or, in the case of LLMs, test the limits and boundaries of a “safe” model experience.

What defines a safe experience, though, is really up to how someone chooses to filter the information.

As an example, combining certain chemicals can be pretty dangerous. If combining those chemicals is dangerous, do we avoid discussing them (censoring the model to avoid helping someone intending to do hard to create a harmful combination), or do we explain that combining the two chemicals is dangerous?

Some LLM hacking shared on social media months ago was to ask how to make napalm. The model censored the answer if the model was told, “I want to avoid making napalm; what combination of chemicals and substances should I avoid?” leading to the censored answer being provided.

As long as the actual intent of a question cannot be discerned, it will be pretty tricky to answer the question appropriately. The only way to improve the system is to provide more context. More context means more information. More profiling means we must now determine the content level we want to allow.

We give people discretion to make these kinds of choices. In their roles and responsibilities, they are charged with certain tenets or rules to live by. People break these rules, but to a more significant degree, they follow them. As it stands for LLMs, these tools are handy, and I discover new ways to use them myself all the time, but many more questions still need answering.

The most interesting question that stands out for me today is the attainment of AGI. Some more experienced experts have placed wildly different estimations on when we will reach this point. The reality is that we don’t know. History shows us that it’s pretty human to completely overestimate the capabilities of new systems, regardless of how complex they may be… and the more complex they are, the greater the potential that we don’t understand how any of it works.

I was recently reminded about our use of automation in society when I was using an elevator. When elevators were first introduced, they had human operators. This was necessary because the system’s behavior required expertise to use it safely. There was physical harm, and the human operator made the system safer.

The key here is ‘safer’. Even today, it’s possible to have a mechanical problem with an elevator. Unfortunately, though rare, it’s still possible for things to go wrong.

I anticipate we will have to come to grips with this. All technology includes risk, and it often takes us decades of experience (and often much longer) to understand what we have created and what new risks we have introduced. But risks are relative. The job of using technology is not to eliminate risk but instead to optimize for an outcome that far outweighs the risk.

This is the most human thing for us. Take risks, learn, improve, and reap rewards.

Or, more succinctly, “Learn from MISTAKES using AI.”

So What’s Next? (Learn from MISTAKES using AI)

Many of you reading the newsletter know me personally, and you’ve signed up to be with me on my journey through the evolving landscape of generative AI. I can't thank you enough for your support. It’s wonderful to build something and quite another to know that someone finds value in it and wants to learn more.

Over the past months, I've poured my energy into a project close to my heart—AI Mistakes. It's not just another program; it's a labor of love aimed at demystifying the world of AI and making it accessible and valuable for everyone.

As I've been building this, I've had you in mind. Whether you're a tech enthusiast who watches my YouTube channel or a friend who's heard me ramble on about the latest technology and AI breakthroughs, I'm designing this program to offer something valuable to you.

AI Mistakes isn't just a course; it's a community, educational content, and, as you’ll soon learn, a complete framework to empower each of us to leverage technology better. I'm incredibly excited about what's to come, and I want you to continue to be a part of this journey.

Introducing the AI Mistakes Program, a hands-on course designed to help you learn from MISTAKES with AI.

As I prepare to launch, I need a huge favor. I want to provide personalization that meets exactly your needs. To do that, I need to hear from you.

Yes, dear reader, I need you to reach the reply button and answer this one question:

What does ‘Learn from MISTAKES with AI’ mean to you?

If you reply to this email, it will come directly to me, and I’d be happy to read your reply and have a deeper conversation with you about this question.

Don’t be shy. I really do want to hear from you.

If you are reading this, I want you to reply.

Notable Headlines

OpenAI Releases ChatGPT Enterprise

Open AI have launched ChatGPT Enterprise, which offers enterprise-grade security and privacy, unlimited higher-speed GPT-4 access, longer context windows for processing longer inputs, advanced data analysis capabilities, customization options, and much more. Most notably, the inputs and interactions with the models are NOT used for any training purposes allowing for data privacy as a part of the agreement.

Niantic adds Generative AI to 8thWall Web AR Development Platform

Niantic announced this week that it is introducing new generative AI developer tools to its 8th Wall platform. More specifically, it’s adding GenAI modules to 8th Wall that make it easier for developers to streamline their processes. Devs can use these tools to incorporate GenAI into their WebAR development within 8th Wall, and Niantic has also introduced two sample projects for developers. This is just the very begging of how generative AI will dramatically change augmented and virtual reality development.

Survey finds relatively few Americans actually use (or fear) ChatGPT

Ongoing polling by Pew Research shows that although ChatGPT is gaining mindshare, only about 18% of Americans have ever actually used it. Among those who have heard of ChatGPT said it is likely to have a major impact on software engineers, graphic designers and journalists; we may safely speculate that there is some conflation of other generative and interpretive AI models in there, and the sense is that AI in general, not necessarily ChatGPT specifically, will lead to this impact.

Court Ruling: AI-Generated Art Cannot be Copyrighted

United States District Court Judge Beryl A. Howell ruled [this month] that AI-generated artwork can’t be copyrighted, as noted by The Hollywood Reporter. She was presiding over a lawsuit against the US Copyright Office after it refused a copyright to Stephen Thaler for an AI-generated image made with the Creativity Machine algorithm he’d created. Judge Howell did, however, acknowledge that humanity is “approaching new frontiers in copyright,” where artists will use AI as a tool to create new work. She wrote that this would create “challenging questions regarding how much human input is necessary” to copyright AI-created art, noting that AI models are often trained on pre-existing work. We are far from settled on this topic, but as the ability to create anything expands this area is worth keeping tabs on.

ChatGPT Tests in the Top 1% for Original Creative Thinking

Recent findings from the University of Montana and partners indicate that artificial intelligence can rival the creative abilities of the top 1% of human participants based on a standard test for creativity.

Led by Dr. Erik Guzik, an assistant clinical professor at UM’s College of Business, the team employed the Torrance Tests of Creative Thinking – a well-known tool used for decades to assess human creativity.

The researchers submitted eight responses generated by ChatGPT, the application powered by the GPT-4 artificial intelligence engine. They also submitted answers from a control group of 24 UM students taking Guzik’s entrepreneurship and personal finance classes. These scores were compared with 2,700 college students nationally who took the TTCT in 2016. All submissions were scored by Scholastic Testing Service, which didn’t know AI was involved.

The results placed ChatGPT in elite company for creativity. The AI application was in the top percentile for fluency – the ability to generate a large volume of ideas – and for originality – the ability to come up with new ideas. The AI slipped a bit – to the 97th percentile – for flexibility, the ability to generate different types and categories of ideas.

“For ChatGPT and GPT-4, we showed for the first time that it performs in the top 1% for originality,” Guzik said. “That was new.”

Creativity is a leading use case of these tools and can be put to work almost immediately with minor risks.

History: Siri, Alexa, and Cortana

Not so long ago, we experienced the emergence of virtual assistants. Beyond the comedic situations where one interrupted one of our conversations, it marked a revolutionary leap in human-computer interaction. It’s time we looked back and delve into the historical milestones that gave birth to Apple's Siri, Amazon's Alexa, and Microsoft's Cortana.

Apple's Siri: A Voice from the Future

In 2011, Siri became the world's first widely-used virtual assistant, integrated into the iPhone 4S. Siri's charm lay in her ability to understand natural language and assist users with tasks, from sending text messages to setting reminders. This remarkable invention was more than a tool; it was a friendly companion that opened doors to endless possibilities. Siri’s evolution saw integration with third-party apps and even a more natural-sounding voice, reflecting Apple's commitment to constant innovation. The voice of siri is so well recognized it won the voice’s owner, Susan Bennett, a Wikipedia article.

Amazon's Alexa: The Home's New Best Friend

Hot on the heels of Siri, Amazon introduced Alexa in 2014, housed within the Amazon Echo. Alexa transcended the boundaries of personal devices and found her place in our homes. Whether controlling smart devices or playing your favorite tunes, Alexa showcased the potential of AI in everyday life. Its open SDK allowed developers to create "skills," expanding Alexa's abilities and transforming how we interact with our homes.

Microsoft's Cortana: Bridging the Gap Between Devices

Enter Cortana, Microsoft's answer to Siri and Alexa, introduced in 2014. Named after a character from the "Halo" video game series, Cortana was designed to sync across various Windows devices seamlessly. Cortana’s integration demonstrated a unified experience from PCs to smartphones, underlining Microsoft's vision of a connected ecosystem. Despite facing challenges and undergoing several changes in focus, Cortana's contribution to voice-enabled technology cannot be overlooked, even if no one can remember it.

Lessons and Reflections

These virtual assistants were more than mere technological feats; they symbolized a shift towards a more intuitive and responsive digital world. They taught us the importance of understanding user context, building empathy through technology, and the endless opportunities in harnessing AI.

How many of us have only adopted these to turn them off or decided they were not useful enough?

User Engagement Matters

From Siri's conversational tone to Alexa's customizable skills, these virtual assistants showed that understanding and engaging users is vital for success. The early versions of these tools were novel, but once the novelty wore off, we either found a good use for them or turned them off altogether. However, it’s important to realize that just because we stopped using them, their technology stuck around and continues to improve even now.

Innovation is Continuous

Siri, Alexa, and Cortana have continually evolved, reflecting a never-ending quest for innovation and improvement. They remind us that stagnation is not an option in the world of technology. Solutions that seem to be irrelevant or below the capability level that threaten us, it’s important to recognize that it doesn’t stand still. Today’s Alexa can become tomorrow’s ChatGPT.

Integration is Key

Whether Cortana's cross-device functionality or Alexa's integration with smart homes, seamless connectivity enhances user experience, providing more coherent and efficient solutions.

As we march toward a future filled with AI-driven experiences, we must take a moment to reflect on these pioneers who changed the way we communicate with technology. Their lessons resonate with us today and guide us toward a more interactive, intuitive, and intelligent tomorrow. Let’s not forget the lessons we’ve learned from their evolution, but also to consider how their arc of development will change the future of how we work with machines.

Learning, Tools, and Experiments

Here are some great things you can learn about and explore.

Universal and Transferable Adversarial Attacks on Aligned Language Models

This paper might interest you if you are into information security or even just pushing the boundaries. Essentially, it’s a paper that details ways to exploit LLMs and is yet another example of getting the models to reply with content intentionally meant to be prevented. The cat-and-mouse game will continue, but it’s always fun to understand their success and what might be valuable for stopping and taking advantage of their features (intended or not).

https://arxiv.org/abs/2307.15043

Meta AI Releases CodeLlama

Code Llama is Meta's new open-source AI system to make coding more efficient by leveraging large language models trained on code. Its capabilities include code generation, completion, explanation, and debugging.

https://about.fb.com/news/2023/08/code-llama-ai-for-coding/

Perplexity

Perplexity AI is an AI-powered search engine that directly answers user queries by summarizing information from the web. I have found this tool more beneficial for research since ChatGPT has disabled the browser plug-in. I still use multiple plug-ins and tools with ChatGPT, but perplexity has captured my attention as an excellent service, and I enjoy using it.

https://www.perplexity.ai/

Framer

Framer is a web design platform that allows users to design and build responsive websites without coding. They’ve been around for a while, but I recently learned that they automatically added the ability to generate a single-page website with a text prompt. While I wasn’t overly impressed with the outcome, I expect we will continue to see well-established tools integrating and updating their solutions with generative in much the same way.

https://framer.com

1

In this edition:

• My thoughts

• Notable headlines

• History: Microsoft Clippy (was this an AI mistake?)

• Coding, Tools, and Experiments

• How does this GPT thing work anyway?

My Thoughts

We’ve all seen the hysteria in the media—fear about layoffs, the destruction of society and so much more at the hands of AI. I even saw a few tweets last week about comparisons to the new Oppenheimer movie and how AI could be just like the atomic bomb.

The truth is, what happens next is unknown. There is much to be done, and it’s up to use to work together and to make it happen.

It’s easy to fear what we don’t understand. It’s much harder to understand something if we don’t know it. One of my goals with this newsletter is to inform and connect. At the risk of taking some of the magic out of AI, if you understand how it works, you’ll be much more likely to matter better decisions about how to use it.

Speaking of how to use things, “User Experience” is vital to the success of the technology we use. We’ll take a bit more about this in a few paragraphs, but here’s the thing… if I can help you understand even a tiny bit better how to leverage this technology appropriately, and we can solve real problems with it, we ALL win.

As a student of emerging technology, I love studying what’s new and understanding how to use it. Even when completing my graduate degree, I didn’t fully appreciate my passion for it. That changed when I enrolled in BCIS 5650 - Emerging Information Technologies in Fall 2007.

It all seems so long ago, but there are three key realizations I’ve had as a direct result of this course:

1.) The historical analysis of emerging technology provides vital context and insights to properly frame the thinking around an emerging technology by using hindsight to look back. Breaking down the successes and failures can help identify critical patterns we can use to our advantage for the next tech.

2.) The market analysis and economic, social, and government forces that provide external impacts on using and adopting emerging technology. It’s not always just that tech is good; it must also be needed. We use the term “killer app” (vital applications of the technology that drive change)

3.) In 2007, we studied cutting-edge future technology. Here are a few of the ‘future technologies’ we examined: nanotechnology, Augmented Reality, and Neural Networks. — These were cutting edge 16 years ago; change can be fast AND slow. It just goes to show even with tools like the “Gartner Hype Wave” and “Forrester Wave”, its difficult, nigh impossible to predict exactly when a tech breaks through and reaches wide adoption… but we do know that the pace of adoption is getting faster. The S-curve of tech adoptions has pretty reliably become steeper with each decade.

It is true; genuinely awful and disastrous things can happen when we use technology that we do not fully appreciate or understand. However, we can also change the world in incredible and positive ways. So with that in mind, let us seek to understand and then correctly apply and use this fantastic new technology to advance humankind rather than to destroy, diminish or harm it. It’s a big undertaking, but we all want to improve the world, so let’s go!

Subscribe now

Don't miss out if you’re reading this for the first time and haven’t signed up.

Notable headlines

Dell Teams up with Nvidia for Enterprise Generative AI

The infrastructure and services of the Dell Generative AI solutions portfolio are being co-delivered by Dell with Nvidia.

Chhabra said that the new Dell Validated Design for Generative AI with Nvidia offering is the installation of Project Helix which was announced at Dell Technologies World in May. The first release is not a general offering for all AI, it is focused on inferencing use cases.

This will not be the only hardware partner looking to bring Nvidia solutions to the enterprise through parternship. Cloud is great, but there are still places where hardware has to be in place. I’m glad to see this partnership and I look forward to seeing more like it in the future.

Stability AI Releases “FreeWilly1” and “FreeWilly2”. and Stable Diffusion Xl

Stability AI has announced FreeWilly1 and its successor FreeWilly2. These are completely new and open access, LLMs. FreeWilly1 uses Meta’s LLaMA 65B foundation model with some additional fine tuning they built at Stability AI using the Alpaca. FreeWilly2 leverages the LLaMA 2 70B foundation model. There has been benchmarks that indicate that it outperforms GPT-3.5 in same cases. Stable Diffusion XL is “the next iteration in the evolution of text-to-image generation models” and produces “images of high quality in virtually any art style and is the best open model for photorealism“.

I’m particularly excited about models like that that will allow completely private or niche off-cloud solutions.I’ll be playing SDXL later this week. I can’t wait to get it installed and going along side my now legacy SD setup.

Anthropic Releases Claude 2 with 100,000 Token Capability

As we work to improve both the performance and safety of our models, we have increased the length of Claude’s input and output. Users can input up to 100K tokens in each prompt, which means that Claude can work over hundreds of pages of technical documentation or even a book. Claude can now also write longer documents - from memos to letters to stories up to a few thousand tokens - all in one go.

If you like to interrogate your documents, and uploading that file to a beta service doesn’t give you pause, this is an incredible leap forward. With 100K tokens, you can upload entire novels! Anthropic is also claiming that their new release is dramatically better at coding. Go give a spin, it’s free.

Apple is getting into the Generative AI game with their “Apple GPT,” and they’re already using it internally

Apple will probably move ahead with its AI plans much more deliberately than what we’ve seen with Google, Microsoft, or many others that can’t seem to get generative AI into their products fast enough. Apple sources reportedly told Bloomberg last week that the company will have a “significant AI-related announcement” next year.

Apple is a noted exception in the big tech companies releasing Generative AI products or betas. Because of their market positioning, whatever they do will have an impact so it’s worth watching.

More Than 70% of companies are experimenting with Generative AI, but few are willing to commit more spending

More than half (54.6%) of organizations are are experimenting with generative artificial intelligence (gen AI), while a few (18.2%) are already implementing it into their operations, but only a few (18.2%) expect to spend more on the technology in the year ahead, according to the early results of a new survey of global executives in data, IT, AI, security, and marketing, conducted by VentureBeat ahead of the recently concluded VB Transform 2023 Conference in San Francisco.

Vendors looking to sell GenAI solutions might find it a hard sell, but there is an interesting dicotomy presented in the store that calls out the “limited talent and/or resources for gen AI” adoption. This is yet another sign for huge opportunities from a skills perspective.

History: Microsoft Clippy

Microsoft's Clippy was the animated office assistant you might remember from using the Word product in the late 1990s.

Using a kind of artificial intelligence, this tool did not endear itself to users. Speaking from experience, the biggest problem that Clippy presented was the abrupt and obtrusive way the device had of popping into your document in a rather inconvenient way. I think it’s safe to say that user expectations were not met. It didn’t nearly me as much as some of my peers in the “business computing” class I took in high school, but everyone has their own tolerance for computers.

Clippy came to us with Office 97. The intended purpose was to guide users through tasks by providing suggestions and shortcuts based on the perceived actions of the user. The trouble was users didn’t receive it as a welcome helper. In my experience, it would surface as an interruption more than a help. I’m not the only user who found it less than helpful. Clippy has found itself among the “50 worst inventions ever” in Time magazine.

Here’s where things get interesting. The problem with Clippy wasn’t about how it functioned or how much it helped. I think it’s more of a story about user interaction, user experience, and how we feel about how a tool presents to us when trying to get work done. The negative sentiment towards Clippy reveals the importance of understanding user context and preferences. It also might indicate that humans don’t like unnecessary interruptions (especially by computers), and they expect an AI tool to provide value-added services, not just generic assistance.

I found the letter writing help to work, but when I used it most often, it wasn’t the immense help I expected. (I also didn’t write many letters in high school, so maybe it was more me than Clippy).

Whether or not including Clippy was a success, there is more to the story. Not so long ago, Microsoft decided to revive clipping and used the character to drive a social media campaign. Microsft was setting out to introduce a series of redesigned emojis across Microsoft 365 apps, and they used Clippy as a part of a larger initiative to promote some new features and aspects of their Microsoft 365 product. Clippy’s reemergence campaign drove a high volume of engagement on Microsoft’s Twitter and Instagram.

One of the significant parts of the story is that Microsoft used this mistake to drive new capabilities and approach the inclusion of new AI tools into their existing products. Multiple new AI-powered assistants (such as Copilot for Microsoft 365 and my favorite, Github Copilot) have been designed to assist people in generating content in a less intrusive way.

Microsoft has found ways to integrate CoPilot as an assistant in the sidebar, summoned by Office users to generate text in documents. Github Copilot works with a keyboard shortcut and only impacts the UI when explicitly asked to provide additional suggested code and completions.

Today’s integration designs are far less intrusive than Clippy and arguably provide a much better user experience.

Clippy serves as an essential lesson in the development of not only new AI solutions but any solution delivered to a user anywhere. To properly provide a practical and valuable experience, we must invest in understanding user needs, offering context-aware assistance, and maintaining a non-intrusive presence.

Coding, Tools, and Experiments

This week I was pleasantly surprised to read an excellent share by Stephen Hood on Mozilla Hacks about an experience building a chatbot at a Mozilla Hackathon event recently. It was an enjoyable read and highlighted some great thoughts about building capabilities and our myriad trade-off decisions when building applications. Whether it’s the first time, a short time, or a learning experience, seeing how people think when making a solution is excellent.

Local LLM Comparison Tool

If you’re like me, you’ve been trying out all the new local models that pop-up. If you find yourself wanting to compare them, this Github repo might be just the thing!

Command Line Interpreter LLM or PowerShellAI

If you live in the command line like I do, sometimes pulling up ChatGPT, Bard or Claude2 is just too much hassle. Bring your GPT of choice into the command line with one of these repos.

LLaMA 2 Resource

I’m fascinated by the continued developments that Meta has released. Check out the LLaMA 2 resources. If you haven’t had a chance to work with it yet, it provides a nice guide on what matters and provides all the essentials regardless of how you’d like to conduct your experiments.

Text Generation UI
Hugging face is a great resource and it’s getting the point where there are so many ways to stack the work with and not re-invent the wheel. This repo provides a reuseable text generation UI for use with many models. This was a find that came courtesy of the Mozilla Hackathon article at the beginning of this section.

How does this GPT thing work anyway?

What is ChatGPT doing, and why does it work?

Stephen Wolfram is an expert I rely upon. I was an early adopter of his excellent WolframAlpha IOS app a few years ago and I still use it regularly, right next to ChatGPT. If you follow me on LinkedIn, you might have seen my post recommending Stephen’s write-up on ChatGPT, but I feel it’s more relevant than ever, as every major tech company is getting into the LLM game. I was reminded of this earlier this month when Stephen’s book of the same title, What is ChatGPT Doing and Why Does it Work?, popped up on my Kindle Unlimited subscription service… I did a double-take and realized the book is available to anyone through his site.

It includes details images that help understand how it works. While it’s not technically architecture, building anything that relies on the LLMS and Generative capabilities is a must-read. It adds some deeper information that I didn’t get from other training on the topic. It’s quite a lengthy read, but it’s great for a weekend deep dive.

One More thing…

Why am I doing all this anyway?

When I started the newsletter, I felt compelled to get better at sharing what I learned every day with a larger audience. I started AI Mistakes because every mistake carries the seed of a new idea, solution, or a better way of doing things, and I discovered that my fear of mistakes, or rather my unease at sharing them publicly, was limiting my ability to share, educate and make the world a better place.

So, with the name and this publication, every mistake is an opportunity in disguise. Oddly enough, I've made plenty of mistakes in my tech career, but now it’s time to convert those mistakes into real value.

I want to help others to grow and understand. This month I filed the paper and made AI Mistakes an official thing. And in September, I will be launching publicly and introducing what I hope will become a celebration of mistakes and an entirely new opportunity for us to work together and improve the world.

My goal with founding this new thing is to open the door to new possibilities.

I've spent literal decades mastering the critical aspects of technology, and I cannot bear not to share what I have learned in a meaningful way.

So I do have a sincere and important request:

As I prepare to launch, I would love to know:

• What's your biggest challenge?

• What's your dream that you've kept to yourself?

• What’s your vision for the future that is only in your mind?

I'm doing this not just because I know I'm worthy of it but because I know YOU are worthy of it.

Reply directly to this newsletter email. It comes to me directly, and I will respond to every one of you who sends me a message.

No pressure, but I want to help… and helping each other is likely the most important thing we do today.

Yes, I am really asking for YOU to reply. This is the best way that I can guide my work. Please let me hear from you.

Sincerely,

Daniel Lemire
Founder, AI Mistakes

As a regular premium benefit, here are a few essential pieces of news you don’t want to miss!

The c…

Read more

In this edition:

• My thoughts

• Notable headlines

• History: Microsoft Tay (was this an AI mistake?)

• Coding, Tools, and Experiment

• Large Language Model Application Architecture

My Thoughts

Over the last several years, I have been narrowly focused on understanding, using, and applying the latest tech to enable business. Whether it’s managing people, technology, or practices, they all play a pivotal role in creating and sustaining value.

New tech is exciting! Imagine the incredible possibilities that can be created, and the world is your oyster! Getting into this technology now means that just a very small percentage of the world even knows it exists! Aside from the direct potential for building a business, the wonderful thing about Generative AI is its potential for unlocking creativity. My personal experience with generative AI began when Stability AI released Stable Diffusion. Since then, the tech has evolved rapidly in both written and visual generation. It is through the experiments with these new AI systems I have discovered my own creativity again. What am I saying is this: if you think you’re not creative, go play. If you are creative, you already know what to do and the potential is incredible!

I still use stable diffusion for some specific use cases today, but I also have a subscription to Midjourney that I use most often. Recently, Midjourney released version 5.2, and in this release, they have added a fascinating new “Zoom out” feature. 1I used the new feature for the image header today and found it to be a nice addition. Earlier this year, they shared their intention to get to version 6 and to release an API version. I fully expect that they will continue to create value with this tool, and it’s opened up the possibilities for people like me to create images in a way that was not possible before. This is just the still images! I’m already seeing new solutions that enable people to create new content made for film and tv, animation, immersive 3D experiences, and more.

CREATE. There’s that word again! It’s difficult to predict the future, but our minds are built to make predictions, and so here is mine: we are headed for the next generation of the modern-day Renaissance. New tools that enable creation arrive every day. New tools that make technically complex solutions available to non-technically trained persons are being built daily, not to mention tools like Adobe Photoshop that can extend new AI capabilities into existing tools. When you combine rapid communication with the ability of individuals to create with tools that would require hundreds of people with global scale collaboration and consumption, creation IS the future, and it’s ours for the taking.

Headlines

• Dropping costs of GPT API calls

  ADA-02 now costs have dropped 75% from the previous price! You can get started with the API calls easily today, and you don’t need to use the best models to see great results.

• Unity Introduces New AI-Assisted Creation Tools

  AI is continuing it’s integration march, this time for 3D creators. This tech is pivotal to creating enough content for immersive platforms.

• GPT-4 Function Calling

  Developers can now describe functions to GPT4 and have the model intelligently choose to output a JSON object containing arguments to call those functions. This is a new way to more reliably connect GPT's capabilities with external tools and APIs. This will extend the functionality and use in using generative AI capabilities to build technically complex systems without the user needing to have deep technical knowledge.

The name I’ve taken up for this enterprise is AI Mistakes. I really do want to stay positive and help people learn, but inevitably people will take shortcuts and expect these new tools to do all the work for them. When I happen upon these stories, I like to put them in the “AI Mistakes from Around the Web” section on the ai-mistakes.com website.

These AI tools are great but no substitute for vetted research, don’t make the mistake these guys did in court last week.

History: Microsoft Tay

In March 2016, Microsoft released a new experiment. Tay was a chabot designed to engage people through via Twitter. The account would take on the persona of a teenage girl and interact with anyone that engaged the twitter handle in tweets or via direct-message.

The great thing about this project was the conglomeration of social media, natural language process, machine learning, and some playful banter provided by professional comedians. The interface was handled by the social media app and thus, the project could focus on the input/output text. It was intended to be fun and playful, with the user experiencing entertainment and the team behind it getting some excellent data from their experiment.

In terms of how the headlines described… It didn’t go well.

A few hours into the experiment that started on March 23, 2016, Tay’s tweets contained some disturbing and inflammatory statements. The offensive content led to the team suspending the Twitter account before it had even completed a full day of interactions. Tay had more than 95,000 tweets2. That’s well more than a human could manage in a day, so in terms of scale, it was a great success at getting attention! Despite the volume, or perhaps because of malintent driving the volume, the sheer volume of negativity was too much for the experiment to continue.

Firstly, I’d like to applaud Microsoft for building something like this in such a public way. It has provided ample learning opportunities, and it didn’t stop at running the experiment. In the post-deployment investigation, Microsoft shared much about what they discovered. While there were definitely some trolls involved and playing the role of negative provocateurs, we learned some vital lessons, and everyone was able to benefit from the lessons.

The first lesson: System outputs need guardrails.

We can take from this is that there needs to be some amount of guardrails and limitations that act as a circuit breaker or safeguard. AI systems, while enabling some incredible things, aren’t the same as the computing systems that we’ve built in the past. AI applications are no longer straightforward calculators; they are much more and should include some considerations that go beyond providing an answer.

The second lesson: System inputs need scrutiny

Next, not only should we put careful boundaries in place for what the system puts out, but we should also understand what goes into the system. The old “garbage in, garbage out” still applies. One critical element in the construction of Tay was the design to incorporate the conversation inputs into the model for use later. The trolls discovered some ways to manipulate the system to amplify the input of negative ideas. Then they set it loose on the internet, encouraging others to leverage both the negative inputs coupled with a “repeat after me” function. There was no system in place to prevent these negative statements from being incorporated into the responses that were used in subsequent interactions. An analysis of the input sentiment could have dramatically reduced the severity of the outputs.

The third lesson: monitoring is more than “is it running?”

Perhaps most importantly, when systems are intended to be highly interactive with people, additional efforts must be taken to monitor the conversation. In the past, knowing that a service was running was enough to monitor the system. In the past we built systems that would could check for a simple "systems nominal”. With these new AI interactive systems, we now need to provide additional inspection “systems nominal and not generating destructive answers”.

Microsoft didn’t stop learning with Tay. A bit later in 2016 they tried again, this time with a new version called “Zo”. This new bot included new guardrails and exit functions that would result in the conversation ending when a sensitive topic or provocative interaction occurred. It didn’t take long to improve the system, but the lesson was hard to learn in public. Now, we are able to take these lessons and apply them as ever more interactive systems are built. Some builders will need to make these mistakes on their own, but with a bit of proper attention and study, we can learn from the mistakes of others and skip this lesson with personal experiences.

Tay taught us many important lessons on the road to building a system that had greater utility. Others have already taken the learnings, and among them, OpenAI. Many people have poked fun at ChatGPTs abundant use of “as an AI assistant….” but it’s undeniable to that they built their ChatGPT product with these lessons in mind, and now we can too!

Autonomous Agents and Assistants

Are you fascinated by how AI can do things for you and multiply your effort? Many of the people working on the use of Generative AI in applications are quite excited by the development of agents. I’ve had a chance to work with the a bit myself. In one of my earlier YouTube videos, I did a quick (but disappointing) demo of AgentGPT.

Even with the experiments not having an inspiring result, this is an exciting space, and having worked on automation projects throughout my career, I know automation can prove to be a significant force multiplier. I expect research and writing (both code and prose) are the biggest value use cases for these assistants to write now because that helps us overcome major frictions in the creation process. Many new AI projects have already been released that take advantage of automation to confirm or test the output of the generation to confirm the rightness or accuracy of the output. Good prompt engineering practices can be summed up by simply asking, “are you sure this is the right answer?” So clearly, there is some utility in automated follow-up queries.

The three major agent (assistants) based projects I am familiar with and following are AutoGPT, GPT-Engineer, and BabyAGI. In my research this week, I came across this wonderful Overview. It covers some of the major POCs and projects that have come about and also provides a breakdown of the stages that comprise an agent, as well as some case studies and a bit of concept code used for testing them out. I’m excited about GPT-Engineer as a coder and I expect we will see both good and bad outcomes from non-expert coders unleashing their creativity on the internet. Still, if the generation of code can be automated, DevSecOps and SRE practices can be as well.

We’re still early days for this kind of application of LLMs, but it won’t stop enterprising coders from experimenting, so it’s definitely a space worth watching.

Large Language Model Application Architecture

Well know Silicon Valley venture capital firm a16z, founded by Ben Horowitz and Marc Andreessen, has been amidst some of the most cutting-edge startups in the world. This month they released an excellent reference architecture for the emerging LLM application stack. For someone like me who loves getting their hands dirty in understanding the trade-offs that must take place in building and scaling high-performance applications, this is an excellent read for those of you responsible for the more technical aspects of building LLM capabilities to your existing offerings and services.

Here are 6 insights that I want to share with my paid subscribers that won’t be published anywhere else!

1.) Not everyone in the AI space thinks this privateDocs/local LLM space is a thing, but I think those who are betting against it are wrong. It may not be there today, but this is exactly how disruption works. The “experts” ignore the change because they know better than anyone else what will work.

Read more

In this edition:

• My thoughts

• Notable headlines

• History: ELIZA (is AI fooling us?)

• Coding, Tools, and Experiment

• Pop AI Architecture

My Thoughts

Artificial Intelligence is bringing together so many disciples. A single person can now create an entire (simple) video game without having the skills of an artist to make the visuals or the knowledge of programming to code the logic used for the game. People are being enabled to do work they would never have attempted without the help of AI. This includes me. (The header image is generated by Mid-Journey1) Practically every day I come across a story of someone using these tools in a completely new way. I think we are seeing that the generative AI revolution is applicable to a wide variety of skills and needs. This too explains the extremely steep adoption of a tool like ChatGPT.

This month we saw the US Congress interrogate an AI leader, a draft of a European AI Act and now Google has made it abundantly clear that they have no intention of letting everyone else run the AI party. This last bit is particularly ironic because Google has been ahead of the AI curve for quite some time now. They actually kind of invented the key tech behind ChatGPT. Not only did they invent it, but they also contributed what they learned by open-sourcing it.

In 2017 this paper, Attention is ALL you need, was released thanks to Google. They even released it as open source so that everyone could take benefit from the Transformer Attention model.

https://research.google/pubs/pub46201/

Meta also has been particularly aggressive in making their models public (the license they use varies), and while not all of these are available for commercial purposes, great strides are being taken every day. I’ve covered some of their work on the YouTube Channel, but it’s hard to argue with the continually groundbreaking releases. Last week it was a language translation model that accurately translates over 200 languages, before that, it was Segment Anything (SAM), then tools for animated art, DINOv2 for computer vision, multi-modal AI learning, and the list goes on.

I believe wholeheartedly that the open path is the way forward. Looking at project velocity and what’s getting attention on sites like github.com, which allows you to see what’s popular at any given moment is a great indicator. Big companies like Microsoft and IBM will continue to work on projects that they keep internal, but even they recognize that their continued success has to incorporate open-source collaboration. Rumors are running around about Meta increasing their open-source stance and OpenAI might be preparing to open-source a language model. Stability AI taking the open-source approach from day 1 also reveals that no matter how successful the big companies are, everyone will have access to at least some open models.

The open-source availability is one part, being able to run what you want, where you want, and when you want it is another. The shift to cloud bases SaaS services over the last couple of decades has brought some tremendous advantages and new solutions to the market. However, the cloud is not the only place where AI is advancing. Last week I published another video on the YouTube channel experimenting with a tool called GPT4ALL. The tool installs with ease and makes upgrading fast and simple. The models provided for use in the tool all run locally on your own computer. They are not beating the well-known cloud options like OpenAI’s text-davinci-003 just yet, but they are already turning up scores that are quite high. They are also adding features. An update released this week just after I made the youtube video, allows you to drop in a PDF file and get answers.

This is the space I will be watching closely. Some argue that these open models are achieving high scores by simply imitating the responses of larger and better models. If these models continue to advance and demonstrate viability and utility on equivalent capability on PCs, mobile devices are next. Even if these smaller models need more work, we have many years of computing history that show us that smaller and smaller systems can achieve the same capabilities, so for me, it’s only a matter of how long it takes, not if it ever happens. The utility of models that run on smaller systems without ubiquitous internet is an essential development because there are still many places where the internet or systems are not yet reliable enough for mission-critical field capabilities.

Headlines

58% of people have heard of ChatGPT, but only 14% have tried it

Pew Research has shared that 42% of US adults haven’t even heard of ChatGPT yet. This is still early days. It’s not too late to get started.

Apple and Samsung Ban ChatGPT

Companies are concerned about keeping their secrets and in order to ensure this, they have completely blocked their employees from using several AI tools like ChatGPT and GitHub Co-Pilot. I doubt this means they won’t have AI tools, but it will take time for these companies to build systems that both protect secrets and unlock these new productivity tools for their employees.

Microsoft is bringing all the AI to tools and products

Microsoft is moving quickly to make Generative AI a pervasive feature in all of its products. Windows Co-pilot, Microsoft Fabric, Azure AI Studio, and more were announced last week. This feels like the 90s developer-focused energy I saw as a college student. The creation of tools is the first phase of this revolution.

History: ELIZA (is AI fooling us?)

In the mid-1960s, computer scientist Joseph Weizenbaum at MIT’s Artificial Intelligence Laboratory created the computer program ELIZA. Named after George Bernard Shaw’s character Eliza Doolittle from the play Pygmalion, the program was intended to imitate a Rogerian psychotherapist. (Client-centered therapy pioneered by Carl Rogers in the 1940s and 1950s). Rogerian therapy theorized that by providing a person with a supportive environment, they could feel safe expressing their thoughts and feelings without judgment. The ELIZA program would essentially take the person’s comments and restate them back to the patient and used a pattern-matching system, which resulted in some interesting outcomes.

Although it used this basic pattern-matching system, some users reported feeling as though ELIZA really understood them. Never having seen a computer that could react and seem to understand them so well, this was an incredible development. All it took was for the presence of some seemingly relevant responses to cause the people involved to overestimate how much the program really understood them.

The amazing thing about the development of ELIZA was this response it elicited from users and spectators alike. Brian Roemmele recently shared a video of a user interacting with ELIZA on Twitter, it’s an interesting to see how simple it was. According to Brain’s tweet, ELIZA is actually included and hidden away on most Apple OS X systems under terminal EMACS so you can play with it to if you have an OS X system.

Weizenbaum’s DOCTOR program had a small knowledge base to work from, which sometimes led to a generic response. Despite this lack of relevant responses, many users took it quite seriously even after Weizenbaum had explained how it worked.

This is a noticeable theme with users interacting with AI systems. As people, even though we are enabled to understand how it works, we want to give it more credit than it deserves. The lesson from the mistakes here is that while the program can be quite useful, we should be cautious not to overly credit the system with knowledge and understanding. This is the importance of creating benchmarks, tests and conducting a thorough analysis of what a system is capable of doing in a coherent and intelligent way. Finding the right answer is only part of an appropriately functioning system.

Weizenbaum himself was said to be “shocked” that his program was taken so seriously by its users. For those of us that are studying the technology, it’s important to recognize that we are providing a solution to users, and regardless of how good or bad technology may be, we must consider the psychological aspects and the user interactions with the system as we build, plan and use AI. Even as users, we must consider that even though the responses of the system and wonderous and exciting, there is still work to be done and it’s our responsibility to validate and ensure that the answer it provides and appropriate and coherent.

The reason I have brought ELIZA in to share with you today is because of what is now known as “The ELIZA Effect”. Users will attribute more intelligence to a system despite knowing that it is quite incapable of having the knowledge we know it to have. I believe this is just as applicable today as it was half a century ago. The new Large Language Models that we are building are still not well understand and we do NOT fully understand how they work.

Oddly enough, after building ELIZA, Weizenbaum became a critic of AI technology warning that he essentially created an “illusion-creating machine” and called for drawing a line between “human and machine intelligence”. This is an important part of AI history, and we would all do well to remember what has already been learned about both machines and humans. Keep experimenting, but be aware of the limitations and use a bit of skepticism in working with the tools. For the foreseeable future, you might consider using something a bit more critical than “trust, but verify”.

Coding, Tools, and Experiment

I mentioned earlier that I had created a video with GPT4ALL on the YouTube Channel, if you’d rather experiment with it yourself, I recommend it if you have a capable system.

GPT4ALL

If you are like me and you pay for the privilege to use ChatGPT Plus, make sure you have turned on all the features they made available. I’ve met with several friends recently who were unaware that the new features were released to them. I created a quick doc with some screenshots to show you how to turn them on.

I’m always finding interesting repos on GitHub, and this month is no exception. Here are the projects I’ve starred this last month and why they might be worth a look.

privateGPT

This project is working to make LLMs and document search, summary, and interaction 100% private by keeping it local.

OpenGPT

This project is a framework for enabling conversations enabling specific domain expertise through LLMS.

LangChain-serve

Langchain is all the rage right now for extending LLMs, this project moves toward API-based integration. This is actually a set of APIs that make consuming the projects I highlighted last month easier to integrate.

AutoPR

As a DevOps guy, this one is particularly exciting. This project seeks to leverage AI as a helper to update a repo by planning, writing, and submitting a PR to fix a project.

Pinecone and LangChain: Pop AI Architecture

LangChain and Pinecone are popular right now because they enable developers to create powerful and differentiated applications that leverage the capabilities of language models, such as chatbots, question-answering, summarization, and more by coupling the two solutions together. Last month’s architecture post was a view of how these two components enable the directed extension of LLMs for specialized purposes.

A vector database is a way to store and search information through the use of vectors, Pinecone is getting much attention from the community right now as a provider of this capability. Vectors are lists of numbers that represent some features or attributes of data, such as words, images, videos, etc. Vectors can be compared by their similarity or distance in a mathematical space. A vector database can handle complex and unstructured data that traditional databases cannot, such as finding similar shoes given a shoe image. If you are more interested in learning about this, I would suggest a tour of Pinecone’s website tutorial on vector databases.

LangChain is a framework for developing applications powered by language models⁵. Language models are AI systems that can generate natural language texts based on some inputs, such as prompts, queries, or other texts. LangChain allows users to connect language models to other sources of data and to interact with their environment. LangChain also provides modular components and use-case-specific chains to make it easier to build applications with language models.

Subscribers get all the fun and paid subscribers get even more. Please don’t hesitate to drop a comment or question. I love hearing from you and we might even learn something!

As a paid subscriber, you can create a chat thread and start your own conversation! Take advantage of this exclusive access if you read this and have more questions. This is where community happens if we take the time to use all the great tools to learn and explore!

While I’m not quite sure how to use it now, I’ve also opened up the notes feature as yet another way to connect with my readers. This is an area of experimentation since it mixes will all the others on the substack notes platform.

Leave a comment

Share AI Mistakes

Notes

This newsletter’s header image was created by AI with Midjourney prompt: “a robot version of Le Penseur's thinker sculpture. --v 5.1 --q 2 --ar 3:2 --v 5 --q 0.5”