The State of Browser Testing - Software Engineering Daily

So when I joined Mozilla - and I've been working on Selenium, I think, for about four years, by the time it became - we started talking about standards. And there was an interesting discussion. I was at a conference in Switzerland. So for the older listeners of this, there used to be a really cool conference called the Google Test Automation Conference, GTAC. And it used to be around the world. And one year, someone from Opera arrived. And he's like, "Look what I've built." And he had built Selenium into Opera. And we were like, "This is amazing. We think every company should be doing it that has a browser." Obviously, I've been doing a few things and I would read like specs and see how that related to my work. Because there'd be times where it's like, "Oh, we're adding this new feature," which normally came from a specification so that we could have interoperability between browsers. And I might help people create testing frameworks around that to make sure that they could test it. But there was never something that was like, "Here's a standard to build a testing framework into -" it was not a testing framework, an automation framework into the browser. And we started doing that. And it was a chap called Andreas Tolf Tolson, who then, about months after that, joined my team and we started building out the specification, everything like that. And I was working very closely with Simon Stewart, who created WebDriver, to get the standard going. He was at Facebook at the time once we got it really going. And yeah, we finally realized how important it was. Because when it came to how standar

So once you've got the idea for a specification, a lot of the work that we did was trying to decide what we could standardize and what we couldn't. For Selenium, the easy things were, "I want to type," or WebDriver. I want to type, I want to click, I want to move the mouse, I want to drag, drop. Those seemed relatively easy. The halting problem, famous Turing problem, and it's impossible to solve, right? And automation becomes so hard to figure that out. And so you're not trying to solve that one. The other one was, is something visible when you want to do it? Because when WebDriver started, Simon was like, "When someone clicks on something, it needs to be visible." What does visible mean, right? And without doing pixel by pixel checks, because you can have overlays and things like that and CSS, has a Z index, but it's not really a Z index because you can have other things that can overlay just by the way you lay out the DOM and the CSS and things like that. And it becomes very hard. And that's where a lot of the effort went into, was trying to solve these slightly harder problems that people didn't need to necessarily solve. And then create a way that people could extend out WebDriver through other standards, through other specifications. So that whenever they were building something, they go, "Oh, w

tration that I think a lot of your listeners will have is that whenever something's going through a specification, it's so slow to actually get specified, right? When I started work, JavaScript was just a mess across the board between different browsers. And you needed to do all these different things, and you would figure it out through callback hell, and you would go through it. And then a lot of the ECMAScript standardization came in. It was like, "Oh, this is really cool." We started working on this, and then they started creating test suites that you could run through everything. But the slowness made it correct. Because if you make things fast, you cause other problems. And we've seen that in the spec world. There was a specification, I can't think of the name of it at the moment, which is used for people to send messages between browsers. So if you want to make a phone call or things like that through a browser, you would do it through this. And it's on its third iteration. And that caused problems. And so that's where it becomes really important to get that correct. It's the same with the web driver specification, right? There are competing tools out there to Selenium, like Playwright or Cypress, and they were never really - that involved in the specification part. But when we've been working on things, we're trying to make it that, "Oh, you can always extend it out," and then go, "Oh, we're going to add this feature to the browser," automatically gets a web driver test. And so you can know that whatever you get from different browsers, and it becomes really important in the mobile space a lot more, because like Chrome on Android, Safari on iOS, you need to think about how those things interact with each other along the way and get the best out for people.

wn away. We've seen numerous stories and it's not actually getting any better, I think. We saw stories about like Replit and things like that last year, who then have putting guards and saves things into that. And then people were like, "Oh, I don't need Replit anymore. I'll just use Claude." And then someone put in something, didn't give it guardrails. And it's like, "Oh, my production database is now deleted. Great." And so I think the AI space is helpful. But I think at the same time, we still need people who understand what AI is generating to be able to give it the guardrails, fix it, solve those problems. Because there's a lot of context that when you and I are talking, that our lived experience will be adding to whatever I'm saying. And AI doesn't necessarily have that kind of thing. But it's also really good at doing quick prototypes or transformations that are really important. So moving data from one place to another, you can go, "Oh, just do this. It's really cool." And

Yes, exactly that. Because observability allows you to create traces, right? And so if you go, "Here's my trace key, and I want you to use it all the way through," some of those things get picked up in how you develop and pass things through. So I've got a whole bunch of talks coming out towards the end of this year where I'm showcasing some of the work that I'm doing. And I think it's got some really important things. Because for years, a lot of people would just focus on reading logs. And you have to know what connection that has, right? But you could always then - when OpenTelemetry came out and people were building it, you could go, "Oh, I could see it go through here." But you never knew exactly how it started. You just saw the data coming into the system. Because then if you see a failure and you go, "Oh, I have this data. Let's see if I can replicate it, how it came in." And if you fill in the form with the same data and it worked that time, you go, "Oh, okay, maybe someone's doing it in a different way." And now you can try to figure out how that person was doing it rather than going, "I don't know how to start this. Let me figure it out." And then kind of spending time. So it's about trying to front load some of that information into your test and then following it all the way through.