Like many of you, we’ve been having lots of conversations about artificial
intelligence and the future of the web. There’s a lot of noise, and it’s hard
to know what exactly we, as web developers, need to know. The web is so much
bigger than our team, so we wanted to gain some clarity around how you’re
thinking about AI, what you hope to learn, and what you want to do with these
emerging technologies. That way, we can best provide you with content to break
through that noise.
Over the last few months, we’ve spoken with web practitioners to understand
the landscape and how you think about AI. Of course, we can’t talk to
everyone about everything. We had just a small set of conversations with web
developers, including web
Google Developer Experts,
focused on how developers are using AI to deliver user-facing features and to
increase productivity in their daily workflow.
We believe that what we learned may be broadly applicable to other folks on the
web. And, we thought the community may be interested in hearing what we’ve
learned.
We took this feedback, along with other research of the web developer landscape,
to create our new AI collection. In this collection, you’ll find
overviews geared towards web developers, codelabs and demos, and other resources
for thinking about AI tools and models on the web.
And this is just the beginning. You’ll see much more from us in the coming
months.
Improved productivity with generative AI
We noticed that web developers want to take advantage of generative AI to
increase their productivity and interact with chat bots to learn new
technologies or seek answers to their web development questions.
The developers we talked to are either already using AI in their daily workflow,
for business or personal projects, or know someone who is.
Code generation
We learned from you that code generation tools, such as Gemini and Copilot, are
great for standard unit tests, basic autocomplete (where you know what to write,
but just need to write it), and simpler functions that don’t require expansive
knowledge of the codebase. These tools tend to be less useful when it comes to
writing more complex, algorithmic code and functions that require broad context
of a specific project.
More senior developers mentioned a concern around the
long-term quality of their codebases,
including issues around code duplication and long-term maintainability. Some
were concerned that less experienced team members may not detect errors or know
how to accurately validate the code produced by generative AI tools.
Developers have also shared that use cases which require specific domain
knowledge, such as writing accessible components, aren’t yet accommodated
correctly by code generation tools they’ve tried.
Learning with LLMs
We’ve seen many developers using ChatGPT and Gemini to learn software
development concepts, like asking a large language model (LLM) to explain how
a sorting algorithm works, ramping up on different programming languages, or
closing knowledge gaps.
You think LLMs provide a great experience due to the quick interactivity of
questions and responses, and because AI won’t criticize the questions being
asked, and just provide the answer they need.
Once again, there is a concern related to more junior developers, where a
minimal level of knowledge is necessary to catch the instances where the model
is hallucinating and generating incorrect responses.
IP protection as a business concern
Many of the developers we spoke with said their companies don’t yet have
policies around developers using generative AI to increase their productivity.
Usage of generative AI tools is often driven by the developers experimenting
with it.
“My company generally misunderstands what AI means, so they’re not
creating the right policies.”
However, businesses that do have a policy tend to discourage usage, due to
concerns around leaking the company’s intellectual properties (IP) to third
parties. There are cases where such policies have been changed, after engaging
directly with the companies behind these tools to understand how the data is
used and potential risks.
With enterprise accounts and partnerships which are dedicated to ensure data
protection, businesses are more likely to encourage developer usage.
Generative AI for user facing product features
On the product side, it wasn’t a surprise to us that, when we prompted our
conversation with the term “AI / ML,” the response was often focused on generative
AI. Developers are curious on how generative AI can be used to improve the
experience for their users, but are unsure about what those experiences look
like and what are the tools available to deliver them in production.
For those developers that have built or are building generative AI features into
their products, using generative AI to answer user questions, with chatbots or
one-off interfaces, is the most common use-case.
Output quality is the top concern we heard from you. In particular, developers
hope to ensure the responses are accurate and aim to prevent the LLM from
generating content unrelated to the intended goal. This is particularly true
when the LLM’s output is directly user-facing, such as a chatbot.
“Demos with AI are wild. Everytime I demo my project, the output is
completely different.”
You’re putting a lot of effort into creating test suites to validate generative
AI outputs for a wide variety of prompts, but there’s no clear and established
way to test or monitor the responses. Most of the evaluation work is manual.
Many developers are new to handling non-deterministic outputs. As a community,
we haven’t yet built systems which work well with them.
The cost of running a generative AI model is also an important concern, and
developers are carefully evaluating the cost against the benefits for their
users.
Standard versus customized models
Most commonly, the developers we spoke to tended to rely on ready-made models
and APIs. This optimizes the time-to-market and the use of engineering time and
knowledge, which were limited.
“I want to stay in the web development space. I don’t want to become an ML
engineer.”
While developers are aware of and see potential value in advanced techniques,
such as Retrieve Augment-Generate (RAG) and fine-tuning, you’d rather focus on
the web development aspect of their work. Ultimately, you prefer to use the
default tools or rely on other teams to produce optimized models for their use
cases.
Privacy and security concerns
Privacy and security were the top concerns, especially for verticals with strict
data requirements, such as the medical industry. On-device AI may be key to
address those use cases, but this area remains largely unexplored.
Exposing user data to more third-parties with cloud APIs is a concern, and a
number of developers see the value of on-device machine learning or generative
AI to mitigate potential privacy and security pitfalls.
AI for web developers
AI is everywhere and evolving at an incredible pace. How can we stay up to date,
incorporate existing tools and models, or work with ML engineers to produce new
models that best suit our needs?
Based on what we learned from you, we’re working on guidance on AI for web
developers. We aim to help you understand AI concepts on a high level, discover
opportunities to use generative AI for productivity gains, and use AI to build
delightful user experiences, using existing tools, models, and APIs. Continue to
check back in as we publish more content in our AI collection.
While most web developers prefer to keep their focus on doing what they do best
(that’s web development!), we encourage those of you who want to dive deeper to
build the tools, models, and APIs web developers need. We want to hear from you
and learn how we can help you be successful.
AI is a fast-moving area. So, we’ll continue engaging with the community as
things change, running more conversations and surveys. If you’d like to discuss
with us, schedule office hours with our team.