[noise] good evening! (bon soir) good evening and welcome to google montrã©al. i'm going to talk in english since the presentation is going to be in english. how many of you is... your first visit to the office? excellent. so, welcome, my name is shibl mourad. i'm the r&d director in the office and i'll give you a brief overview of the office. we have a small office (by google standards),
we have around 50 developers... software developers and business developers in the office. we work on chrome and web security and we are constantly growing the office, so we are looking for software developers who... in many areas - machine learning, windows development, front-end development, server development... so, we have james, who i don't see, but he is our recruiter, and if you see anybody with a t-shirt like this you can ask them questions
about joining google. there's one guy here. i'm happy to have alex here and this is the end of a three-day visit of alex to the office. so, alex has been giving us training about windows development in the office. he has a long background and long expertise in windows. he has written a book called windows internals - which is the bible - and he's going to talk to us about reactos, which is an opensource windows platform. that's pretty amazing.
just to give you like an idea of how good alex is and i don't say this for everybody, but this is the case. he has been training our software development team and the team in montreal is actually one of the expert windows expertise teams at google - i mean chrome in particular and when they asked me they would like to get alex as a trainer i said "well, come on, you guys must know everything. why do you need alex?" so after the first day, i ask people "how it was?" and many people said "i have a headache", so this guy is good.
so, i'll leave it to you, alex. - alex: thanks! ...if this works? hello! ok, i'll stick with this. alright! so, thanks for coming. a little bio about myself. shibl said a lot, but i'm actually a local montrealer so i studied at concordia, just a few blocks away from here. who concordia? no no no, yeah, it's what i thought. i started working on reactos in 2004, i was still in high school i think,
and basically discovered this project, or, these people who are trying to re-implement windows nt and i had a really big interest in operating systems and i looked at linux which was opensource, but had a very tough learning curve. it's hard to get in a linux kernel i think when you're you know, 17 or 18. looked at mac os which is opensource, but you can't really send major patches to apple, and reactos which is windows, but not really windows. so i though hmm, this is interesting.
so i've got involved with that and pretty much spent 5-6 years working on it while studying and studying less and less and coding more and more on reactos. and then eventually i left montreal to go work at apple, i've actually been an intern there for a few summers. logically right - you work on a windows clone and then you go work at apple. apparently microsoft didn't think i was good enough. so i worked at apple on the ios kernel and
on the ios platform and then work went off to start-up in the bay area. ironically, while working at apple, i was approached by people from microsoft to write a book on windows, which is called windows internals. so i was working at apple writing a book on windows and then microsoft asked me to do training and go to their campus and teach their windows developers how windows works while spending my nights reverse engineering how it works to undermine their
operating system to make a free open-source version of it, while working at apple. so i got really confused and i kinda left reactos for a few years and only recently came back to it and realized... ok, now i know where things are. i'm no longer at apple, i'm doing less training at microsoft cause google pays better and i can do other things. so, what is reactos? reactos is an opensource re-implementation of windows, and not windows 95, but windows nt
which is how we call the family of windows operating systems with nt 4.0, windows 2000, xp, vista, 7, 8, 8.1, so none... none of the dos-style windowses. reactos tries to reimplement windows nt 5.2 which is server 2003 or xp 64-bit, but not 32-bit. i'll explain why that's actually important difference. the kernel is 100% written from scratch, so it's not based on unix, it's not taking windows applications and
emulating them - it's actually running a windows kernel that we wrote and running windows dlls and windows applications. so it has the windows boot loader, the windows kernel, the windows dlls, the windows drivers or reimplementation of those things. the hope is, you can take any windows binary or take any windows hardware that you have and just run it in reactos without having to have a windows license and with being able to see what is the os doing. learning, not having a binary file, but actually having c code
that you can look at "hmm, how does windows create files?" well, i can go look at reactos and see how that works. so, everything from the boot sector down to the data files that contain language strings are all trying to reimplement this windows behavior. now, a lot of people would ask "why are you wasting your time reimplementing windows? don't you have a life?" well yeah, probably not. but! for myself and for many other developers, especially when you're just
young and starting out, it was a really good way to get exposure to windows. windows development is something that you don't normally learn in school. windows kernel development is definitely something you never learn at school. so the chance to actually play around the source sandbox, try to implement a windows api, try to see what happens when i try to schedule threads this way and being able to boot into something that looks like windows and works like windows,
that i don't need to learn how linux works, i don't need to have a mac, i can just take the reactos, boot it, kinda see what happens when i change an api around or when i do something different. or let me try actually implementing some functionality. maybe i would do visual basic app when i was young, it's not working on reactos. let me try to make it work. so it's a learning platform, i think, for a lot of people, to give them a chance to work in a windows environment, whether they
want to do kernel-mode stuff, whether they want to do the user-mode, whether they want to do a simple paint application, because reactos is not just the kernel, it's... everything that windows has, reactos also has to have. there's also other reasons why some people work on reactos. one of them is they're actually hoping to really get rid of microsoft and make sure that everyone in the world will be using reactos. i don't think that's ever gonna happen.
but, some people can dream. and there are actually legitimate cases where, for example, if you thought about companies where they sell movie ticket systems, you go to movie theater at paramount, you try to buy a ticket and all they have is a little app that lets you buy ticket, but they're running windows because someone wrote it in visual basic or java or whatever, and they're paying microsoft millions of dollars for thousands of ticket machines around the country to run one app that prints a ticket. a lot of smarter companies would probably have web app, run linux on it, but some companies just can't move away from
windows. so hopefully, if they run reactos it installs the same, it looks the same and it runs the same app, they can save that money. now, that wasn't my motivation, but it's something that you can do. and i know another use of reactos is companies that are building windows products and their products are hitting bugs in windows or they aren't working like they'd expect to. the chrome developers are trying to get windows to do something
and windows doesn't do it right. well, with windows all you have to stare at is a big binary file and you have to reverse-engineer it to find out why is windows failing. with reactos you can see what is the actual c code, human readable comments, human readable code, they're gonna tell you "here's what windows tries to do." so it can be a great help for the people that are actually developing for windows. and another use which i've seen is, let's see, you're building a windows driver
and you have all these windows functions that you're calling. windows has a lot of nice libraries - for compression, for building a binary tree, for scanning through a string and converting it... if you take your driver and you put it on linux, you don't have the same functions. you maybe have alternates and other times you have to re-implement them. but if you actually take a part of reactos, link it as a static library, add that to your linux driver and now you have the same functions when you're running on linux as well. so it can help with cross-compatibility
as well. so, why nt 5.2? why server 2003? so, keep in mind this decision was made in 2006. so xp was out at that time and xp was a really popular os. it took the really good design that nt and windows 2000 had that was really solid and scalable, and took the windows 98/95 kind of friendliness that users liked. so you can have the best of both worlds, you get the good design that wasn't based on dos, and you had directx, you had the hardware acceleration, and you had games
and xp is still still so popular that people still use xp today. it's been a twelve-year-old os and it's still one of the most commonly used operating systems. but xp wasn't perfect and so, before vista came out, microsoft worked on an update of xp for server systems that they called server 2003. and that's on actually a fairly different kernel, fairly different semantics, they improved it a lot, but they only have released it for servers. now, the 64-bit version
of xp which came out a few years after xp had this too, but noone really ran xp on 64-bit. so very few people actually know about server 2003, that was as really good os as was xp, even better, before vista. because then people remember vista and... we know what happened there. windows 7 cleaned up a lot of that and today if i wanted to start reactos, i would probably aim for windows 7. but that decision was done in 2006 and we picked server 2003. mind you, there are very little applications today that won't run on xp. pretty much everything runs on xp,
even more stuff runs on 2003, very few apps require vista or later. so it's not really hurting us they were targeting this fairly old and legacy operating system. surprisingly enough, reactos is actually an almost 20-year-old operating system. these guys got around in 1996 and decided to start a freewin95, a free version of windows 95. after a year they gave up and they started reactos, which is a free version of nt4. then they had two developers that wrote about a 100 lines of code every week and microsoft was releasing new versions of
windows faster than they could keep up and then they can't keep up, there's only two of them, so nothing really happened. 2004 is when reactos first had a gui. and that's when i discovered it, that's when i saw the slashdot - hey, here's a windows clone and there's actually gui, there's something behind it! before that there was just kinda text. and so i joined and from 2004 to today reactos went from 20,000 commits to 60,000 commits. and you'll see awesome code metrics where
project really took off because people started seeing the gui, people start seeing the future behind us and a lot more developers join, but a lot of people call it a 20-year-old project because of that old legacy, you know, freewin95 story. this is how reactos 0.2 looked like. this is where maybe you wanna say i wanna dump all my time in this, this looks cool! and you'll see the demo looks lot better than there but it's impressive that just a team of 4-5 developers actually got this far.
so actually have and these are actually windows applications that are being run, rendered horribly, but they are running. so reactos is very, very big. it's 6.25 million lines of code today and 60,000 svn revisions. in fact, other than haiku, which is a opensource version of beos (which you've probably never heard of beos, old, but really cool operating system) reactos is the only opensource os project. if you look at linux, linux is a kernel project.
linux isn't work on all the libraries and all the apps that ship with linux it has lots of libraries, applications, shared libraries, drivers that other people work on. but linux project is mainly the stuff in the kernel. now, fedora and ubuntu, they're mainly taking everyone else's code and packaging releases, which is hard. it's hard work and they have to do it. but there aren't many projects out there that are reimplementing an entire os. every library
we write, every little application, reactos notepad, reactos calculator, reactos paint, we write the kernel, the drivers, everything has to be written from scratch. and so this actually puts an incredible strain on the tools that are out there. you tell a compiler to build an os, it's gonna freak out. like the first time we got visual studio to load a solution file for reactos - it crashed! "i wanna have nothing to do with this 6.25 million lines of code
project among 500 binaries" and a lot of other opensource projects that help and a lot of tools that build the reactos, actually, we found bugs in them because of the huge size of this project, and the huge depth of it. we'll go over some of that. it's actually surprisingly popular, depending on what your definition of "popular" is. every release gets about a million downloads, so we release about every 2-3 months
and million people are actually interested enough in trying a barely working version of a windows clone that's based on xp. that's kinda cool. we got nominated at and won sourceforge "project of the month" a few times, we participated in google summer of code for two times and we keep being invited at various conferences to speak about what is we're trying to do. which is surprising because the linux community and the opensource community doesn't really like windows, they would rather see windows die. so the fact that they actually care about a
reimplementation of windows is somewhat surprising. we're a community of about 200 people on irc that are hanging out there, 50,000 people on the website, forums and the mailing lists so people are always kinda keeping in touch. in fact, there's actually a school out there, that studies os design based on a book some chinese guy wrote, called "windows kernel analysis with using reactos" and it's 2,000 pages of reactos source code explaining this is how an operating system works. so, an entire army of chinese students are looking at my bugs,
thinking that's how os is supposed to work. scary... in russia, they just launched a course taught at moscow technical university that's an os course and it uses reactos as an example of how an operating system works. putin received a private demo of reactos and has a picture of going thumbs-up. it's really freaky. we actually have the reason we have staff in russia, in china and all over the place. it's great, actually, to have
developers all over the globe. we're about 30 developers that contribute, some from time to time. we're about 150 developers that have developed some code at some point. it's also fair to see people leave for 4-5 years they come back, they leave. we don't actually have - we don't pay any of our developers. i never got paid, noone else got paid and so any time we spent on reactos is just basically time we donate and things that we like to code on
we've got people on high school, we've got people that're sixty, they're senior managers in a company and from time to time they want to write some windows code for reactos. so, there isn't a lot of... there isn't, like, a seniority level, there isn't a lot of big coordination. it's a lot of ad hoc and collaborating with 30 people across 12 different timezones is fairly hard. in fact, we actually have on the reactos page (that, by the way is what happens on reactos.org when you put in a wrong
url, it gives you a fake blue screen of death) i'll be on the people page and i click on the "developer". we have developers in europe, asia and all over the us and canada. and if i add language maintainers and moderators i start having someone in india and if i add newsletter authors and translators, it's gonna open up even more people and if i add users... pretty impressive, we pretty much have users all over the world, except antarctica. so, noone's paid and it's really interesting to see
basically random people on the internet working together, trying to work on this. we do have a monthly status meeting and that's kinda where we try to set a direction, set a tone, that people really come to reactos and they feel like reimplementing an open version... an opensource version of paint. well, go ahead and it will be reactos's paint as long as it looks and behaves exactly like windows's paint. so, you're reimplementing all the windows from scratch behind microsoft's back, what could go wrong.
i am not a lawyer, i know that reverse engineering is supposed to be legal in some countries and is not illegal in many countries. now, "not illegal" doesn't mean "legal", it just means "not illegal", as long as you're not breaking any drm or cracking. but there are patents, right, companies can patent algorithms, they can patent the way the function works and if you're reimplementing it, even if you're doing it from scratch, even if you're doing with your eyes closed, if it does the same thing the other algorithm does, you can get sued. now, some countries don't have software patents
and who knows what microsoft patented. in fact, i don't wanna know. patent law says if you don't know, you're less... you're gonna pay less. if you know you're violating something, that's worse than if you don't know. so i don't know about any patents. so if you ever wanna license, make money off, start a startup on reactos, go see your lawyer. i'm doing this for fun. reactos is basically a huge project and it needs a lot of infrastructure. we have 4 servers in sweden that some university student donated to us, now on his campus,
2 in germany, 1 in poland. so all our services are running on that. we run atlassian tools, so if you've ever worked with jira, crucible, fisheye, that's what powers our development. we use buildbot for building (i'll show you what that does), cppcheck, we use doxygen for documentation and we have a whole suite of regression testing tools that are custom made to build every version of reactos, run tests (and it's quite interesting in how you test a whole operating system) testing
now, we use subversion and we also have a git mirror and a code.google.com mirror. online we use drupal, iserv, mediawiki, nginx, phpbb, so we have a whole team that knows nothing about os development, but their job is to keep the website alive, the forums, maintaining servers... so like i said, in the end, whether you're a php developer or linux kernel developer, there's probably something for you to do. so just maintaining all these servers and all the website code and all the
databases requires a whole team. so i wanna show you one of the really cool services we have that are actually making life of developers and users easy. so other than people map... that's our jira page, that's where i log in every morning (well, not every morning, only when i don't have too much work in my real job) see all the bugs that are assigned to me. something about the memory manager not making any attempts to reduce memory consumption. eh, who needs that. the more important bug to me is that
minecraft does not start up. that is a big bug - people would use reactos if we ran minecraft. so, jira's an opensource - not opensource, there's a little star there - but it's basically a replacement for bugzilla. let's see... trying to find... so, every build goes on iso.reactos.org so we have a server here that has every single build since 2007 or 2006 which is great for regression testing - cause someone can pickup a build from 2007 sends a bug report that says "this used to work in 2007, it doesn't, fix it"
someone has to find out in last six years which build broke that person's app. it's not a lot of fun. that's our forum... actually, let me just put the website directly, it'll be easier. ah, there we go - testman. so, every time there's a build, a bunch of servers build it, open up a virtual machine and test it. and, in fact, we build it with visual studio, and we build it with gcc, that's something i'll talk about. and then we test it in a virtual machine, we test it on a real machine and we test in a different virtual machine that's made by another vendor.
because, it turns out, when you're making an operating system, just like when you make a web page, different browsers render it in different ways. well, different virtual machines run the code in a slightly different way, different compilers will actually build code in different way and sometimes you can have bugs due to the compiler. and so these're the last few revisions that got built and then they're all tested with an extensive suite of testing tools that i'll mention. but i can pick these two versions,
compare the results and it's gonna show me all the tests we ran (we'll talk about the tests a little bit after) ...the machine is lagging... ...not making a new window there... and there i can see how long it took to boot, how many context switches, how many interrupts, and what tests... what the test results are. obviously, red is bad, green is good and i can look at what actually changed between those two builds. and between these two builds
quartz:filtermapper now runs two less tests so for whatever reason it's running less tests than before. that's usually not a good sign, that means the test crashed. and this other thing which crashed and wasn't even able to test - is now running 240 tests, so that's good, the other one is not so good. and so, hopefully some developer's looking at what did he change in this build to make that thing better, but the other thing not as good. so this is how we keep track of how we're
making the project better or which is making things worse and worse and worse and worse. we also have a getbuilds system ...getbuilds... ...oops, getbuilds, thank you google... where you can just put any revision you want and get a premade bootcd (= livecd) and installcd (= bootcd) so you can actually try it out. so you don't have to check out the code, you don't have to be a developer, you don't have to compile it by hand, any build you want, you can run it.
so for people who are doing testing for us that's great cause if they find a regression, we tell them to test for revision in the middle and they can just get a new build, try it out, boot it (as we'll see how to try it out) and fix it. so we have a whole bunch of services that people created on the website that have nothing to do with windows or with kernel writing, but they're really making our job easier. i'll also show you buildbot. so, if i go to build.reactos.org
...oops, build... to waterfall display and i'll be able to see all our servers, what revision they're currently building, if it managed to build it or not, any warnings that came through from that build and then i'll be able to see it testing it and what the results of the test were, so if something did fail during a test, i can see the whole log and everything that happened. and this is good to catch developers that are
checking in broken code that doesn't build, or developers that are adding regressions. so, up in 2011 reactos built with gcc and mingw. gcc is the opensource gnu compiler and mingw is the cross-compiling kind of patch and toolset around gcc that lets you build windows applications with gcc. now, turn's on actually constructing a buildsystem - instead of using make, we decided to make our own buildsystem, called rbuild. turns out a team of os developers does not make a good
build system. so around 2011 we scrapped rbuild and switched to cmake and cmake is fricking awesome. because cmake not only allowed us to build with gcc, but it also allowed us to build with visual studio. and we first had visual studio builds and clang builds (which is kind of apple's compiler these days), and visual studio has a static code analyzer and clang has an analyzer too. so basically you get more code coverage, see what bugs we have and the more compilers you build your code with, the more you're gonna find errors,
that's a truth. and we use a project called ninja (which is even more fricking awesome) which basically makes our cmake build 10x faster. ninja is really really cool. so, once we had visual studio, the really nice thing we had, as we now had pdbs, which are microsoft's symbol files that let you debug stuff that you're building with microsoft tools. when you're building with gcc, you have the opensource tools, but they're not really meant for debugging on windows, especially not an operating system.
so we had our own custom debugger, we had gdb support, but it didn't really work that well. with visual studio builds we can actually use windbg, which is the windows debugger. and you can actually use windbg to attach to a reactos instance that's running in a virtual machine, just like you would attach to windows operating system. you can use the same windbg extensions that microsoft wrote to analyze hangs, deadlocks, issues happening on a reactos machine
just like you would on a windows machine. in fact, when a reactos machine crashes now and you connect to it with windbg, windbg will analyze the crash dump and ask if you wanna send it to microsoft, so they can analyze it. and we always click "yes"! and i have friends who've actually told me they get these dumps which look like windows, they start looking at them and at some point they go "what the...". so we like wasting your time. so, the way we test reactos is two ways - first of all, developer-centric testing, so we write unit tests, functional tests, we have a whole testing suite that i'm going to talk about.
the other kind of testing is with all these users who are trying it out. running their favorite app, trying their favorite thing, and finding issues... oh, back. try to run windows, try to run mac, try to run linux, there aren't a lot of operating systems out there. and so if windows, mac and linux are all accessing hardware in a certain way, a virtual machine developer might think that that is the way that they're supposed to return the output. if reactos does something differently (which is how
the intel cpus are supposed to do it), but a virtual machine ended up depending on how windows expects it to do it, we might actually find bugs in virtual machines. so, we found some of those in qemu, which is an opensource virtual machine project, we actually have a developer that left reactos to work on qemu, and we've found bugs in virtualbox as well, which is a sun sorry - oracle virtual machine and we sent them patch as well. so we find bugs in vm products as well. switching to cmake and ninja made us one of the biggest projects in the world that uses cmake and ninja
and we also found some bugs in those, so we sent patches as well. so just because we're taking a lot of projects that were not meant to deal the size of an os, and also because we're taking all these opensource libraries and compiling them with visual studio or running them in a windows-like environment, we're finding bugs and hopefully sending patches. so it's been interesting to see the kind of collaboration that we can have. so today reactos has a kernel that's about
80% complete. so kernel-wise we're actually doing fairly well, the memory manager is the one thing right now that's undergoing the really big rewrite, because it's originally based off some linux concept and some general computer science concepts, but not around the real things - the real complications that nt brings to it. now, the plug and play manager's also fairly custom. so two big things in kernel that are not really compatible with nt right now are memory management and plug and play management. the security subsystem works, but there's a lot of things that are not working
right. so up until a few years ago actually any kind of access check always said "yeah, you can access that". it wasn't the real thing you want it to do. but scheduling, interrupts, process management, device driver support inter-process communication, all these things are binary compatible with windows. and it uses source compatible to windows. in fact, someone asked me earlier during the training "do you use the same function names?" yes. "the same structures?"
yes. and so, when i show you windbg... windbg actually is the windows debugger that relies on certain structures being named in a certain way, fields being in a certain location. and the fact that windbg is actually fooled to be able to debug reactos shows how close we are to that. in terms of other than the kernel, we add all the drivers and we support a lot of drivers today, and so sound card drivers work fairly well, network drivers work pretty well as well, video card drivers work,
but some things we just don't support yet. like, there's no printing support, usb support is iffy and other things like wi-fi support... i think we support one wireless card. so, we're supposed to be compatible to drivers, but a lot of drivers use functionality, calls and behaviors that we simply haven't implemented yet and so we're working on getting there. now, the window management is the nasty part. the window manager we have is based off wine and wine is based off talking and converting the window manager calls
to x calls. so wine doesn't really do any window management. it relies on the linux window manager to do everything. so we have to write our own window manager which is compatible with wine's window manager which isn't really windows's window manager. so we're in a kinda bizarre world where we try to make things compatible, but use as much wine as possible. so our window management, which is called win32k.sys, is fairly different from how windows implements it internally. which is probably a good thing
because if you've ever done windows security, if you look at the patches that are coming in every month, there's gonna be about 150 exploitable kernel bugs in the window manager windows abi every year. so, thankfully those are not our bugs. but a lot of security companies are coming to us and saying "hey, can you hurry up and make it more like windows? because then any bugs you have are gonna be bugs in windows too and those are worth a lot of money". and other
than that, in usermode most the other dlls are from wine. you know, wine has a lot of support and wine can run almost 20,000 windows applications today. so we should be able to run everything that they run and if we're not able to do that, that's because we have some bugs or because the wine dlls are expecting to be running on linux kernel. and so because wine wants to run on linux, sometimes the way they do things, sometimes the way they do xp things... it's not really how windows does it and - sorry... it's really how linux does it and not how windows does it. so whenever that happens, we actually
have to fork their code and add differences to make it work on windows. just recently, about two weeks ago, we were actually able to get windows session manager (smss.exe), the windows subsystem (csrss.exe), the windows subsystem server (csrsrv.exe), ntdll.dll and kernel32.dll, just some of the most core windows binaries out there, copy them on top of a reactos installation, reboot, cross your fingers, nothing broke. it still booted,
the apps still worked, and so we got to a level and we were really shocked. we took microsoft's own dlls, put them on reactos and they didn't freak out. and so we got to thinking, we have all these tests that check kernel32 abis, and right now we have about 40 failures. if we take the windows kernel32 and we have 30 failures, that means that 10 of those failures are probably actually bugs in kernel32, but the other 30 failures, they're happening even when running microsoft's kernel32.
so now we take the test and we run it on windows and we get 0 failures. which means those 30 failures are not actually caused by bugs in kernel32, but by something that kernel32 might be calling. this is the greatest way to find where our bugs are, starting with windows binaries, drop them in and see what happens in the regression tests. see if we're doing better or worse, we'll probably be doing better. but if the count of failures doesn't go to zero, that means the bugs are not in that module, but something in the module uses. and so if we find out everything that kernel32 uses and slowly replace all those
things, the failures will go down. and so we actually started fixing a massive amount of bugs in last two weeks by saying "ok, we start with 30 failures, let's try bringing this in. oh, 29! so this thing has one bug at least that's causing this thing to fail" and we start debugging, analyzing and we find the bug a lot easier. so we're actually doing a lot of effort right now to try get windows binaries, just drop them in and see what happens. and of course the real cool feature will be when you take
reactos binaries, drop them in a windows installation and it just works. because those reactos binaries will have source code and symbols! and so if your app crashes when you're calling a windows api, if you're using the reactos version of the dll and visual studio, you'll see the c code. and you can actually see "oh, it crashed here because i use a wrong parameter and there's even a comment that said - using wrong parameter and we still have to reverse engineer the windows dll". so, as i mentioned, wine can run 20,000 applications.
we can run maybe... a hundred? and so anything that wine runs that we don't must be because we have bugs in our kernel, in our drivers and in all the dlls we're not taking from wine. so, hope is to keep working on this and find these issues. then a lot of times an off-by-one bug in the kernel can cause a dll not to work right and a 1000 applications that are relying on that not to run. and so it's always nice if you fix a tiny little bug and then suddenly office loads. so once we fix the memory manager,
the cache manager, we actually figure we're gonna get a lot closer to where wine is and once we fix the plug and play manager, we actually believe that we're gonna get almost complete compatibility to windows drivers. because windows drivers use object management, memory management and plug and play mostly, and we have all of that other stuff. plug and play is the one thing that we know isn't really working right. so once we fix it, that should just help out a lot with the drivers. fixing all those things will make our backend code very compatible, but at the end of the day the biggest bugs reactos has
is how things are rendered. and things aren't rendered that nicely because of our window management. we have subtle timing bugs, when applications are sending windows messages between each others, and we have not so subtle horrible rendering issues. someone asked me at google "can you run chrome?" and i tried it and it runs and everything just renders as one pixel. so, we have a lot of work in that area. so that being said, you can help!
we actually always welcome new developers. we have a whole team not just of developers, but also web designers, testers, web artists that actually make the art, the icons (and we will talk about all the icons) and the strings. windows supports hundreds of languages. reactos you can install in catalan, in basque, in spanish, in frech (not yet in quã©bã©cois, but, you know, maybe one day), and we have people who do nothing else but translating reactos, not the source code, but the strings in their language.
we have people doing pr, we have people who are just enthusiasts, and of course we have people with money, giving us donations. you can always help that way too. and you don't have to be a kernel coder. i'm a kernel coder, i mostly work on the kernel, but if you are just interested in making an open source version of photo viewer or some bizarre windows app that ships with windows (so not calculator because we have that one already), but sound recorder. we don't really have a good sound recorder app. we have some player, but not sound recorder. you can be the one who wrote the reactos
sound recorder as long as it kinda looks like windows's, cause we care about that. i mean, that's good! you can take a tiny little dll that windows ships with that has 3 functions, like compress.dll. and you can try your hand at implementing compress.dll for reactos. you can take calc and maybe calc doesn't do square roots yet (it does, but let's say it doesn't), you can be the one that added the square root function. you might be able to find a tiny bug, tiny typo in some kernel function. maybe it has nothing to do with being a kernel developer,
but there's an if-statement and instead of having a brace it has a semicolon and so the if doesn't really execute and the semicolon always executes. and you fix that bug and because of that firefox runs better. in fact, you don't even have to be a c windows developer. you can be linux php developer, as all our infrastructure runs on linux and all our backend stuff runs on php. and you can be an artist. all those little icons that windows has from the start menu to the little windows logo to all the other things - we need people to create
that. those things are copyrighted, we can't just copypaste them, and you can't reverse engineer an icon. you're copying it at that point. we actually have a fundraising campaign going on for €30,000. euros because reactos is an actual foundation, non-profit foundation in germany and we've actually got €15,000 so far. so other than paying infrastructure this money's not going to pay any of our developers.
but still, we have contracts. so if some things that we simply don't know how to do, don't really feel like doing, and maybe don't have the skills to do, but maybe other people do. maybe they're lacking the motivation to do these things, but for a little bit of money... maybe they'll do it. so we have little contracts that say "you know, if you do this for us, we'll pay you this much money." now, if you're working for google or a large company,
these are probably not gonna be worth it. these are really meant for students that are trying to get some experience, but they're lazy and they need some cash. like the thing - i have one example here... shell development contract. so, if you improve our shell32 library, so we can run... so we actually had this guy that wrote a clone of explorer, the shell, the start menu, the taskbar, the desktop and everything, and it worked fine in windows, a 100% explorer clone. but that's cause in windows you have a library called shell32
which gives you most of the shell's functionality. in reactos our shell32 is not that good. so his explorer doesn't really work. so now we have to fix our shell32 to run his explorer. so we're gonna pay 600 euros for 100 hours of work! which if you're a sixteen-year-old in europe, it's not that bad. spain has like 30% of unemployment. they can go work on that. so again, you're not gonna find thousands of dollars here, we're not like a megacorporation, but,
you know, some money on the side for your little brother that know a little bit of coding, why not. so, if you wanna reach out, see more about what's going on, talk to us, on reactos.org is where we have a forum, mostly for users to talk to each other and enthusiasts who kinda try things out. we have an irc channel on irc.freenode.org where a lot of the opensource projects hang out in - there is #reactos and #reactos-dev
and pretty much all the developers hang out there, at different timezones, a lot of them are from europe, some are in asia etc. and we also have a mailing list mailto://ros-dev@reactos.org so i think this is getting recorded, so if you don't memorize these things, you are ok, though you can probably google for reactos, it's not really that hard. all these things are on the wiki, i'm alex.ionescu@reactos.org, i'm also here, so if you wanna talk to me after, you can ask me questions. and if you're in any way interested,
any of these channels are good ways to get in touch. so i actually want to go ahead and demo what this thing looks like! does it really work or was it just lying all the time? so, i'm gonna use virtualbox which is an opensource- well, it's not really- half of it is opensource. but it's basically a vm that lets us run reactos. so i'll just run it, a pre-installed version, then i'm actually gonna go through it and show you how the install looks like. so i will make it fullscreen... so this is the boot- resolution's kinda weird,
bootmanager... so if you've ever used the old nt loader, that's what you get in xp if you press f8 at boot-up time, so i tell it to boot reactos... i need to fix... i won't make it fullscreen because otherwise you won't see the whole thing. let's see... oops. let's do that again... let's just do another reboot, just so you can see a fast reboot cause it's really impressive. alright,
so this is kinda from my bootcd, i don't wanna boot from the cd, and... and that's it! it'd actually play a boot-up song too, which if i plugged this thing in it, you'd hear. so, here's... the lag is because this is like streamed on google hangout and streamed back here and streamed to, like, mountain view or something, so, it's not- the mouse is not actually looking like that, trust me. start menu, programs,
explorer, games, it's still one thing that everyone likes to do. solitaire, minesweeper, yup. so for the company out there that's paying windows licences so their devs can just play minesweeper, we have minesweeper. we don't have hearts, though, so anyone who wants to write hearts, that'd be great. we have solitaire, we have spider solitaire, command prompt... so let's actually try to go online. so, one thing we do have that windows does not (and we try to stay away from those things, we don't wanna...
we don't wanna do things that windows doesn't have built in), but because we are aware that there's a lot of things that we don't ship with, like, we don't have ie and we probably never will have ie. but we have an applications manager which is similar to an app store, basically, and i can go on internet & network and download bittorrent, of course, we fully support bittorrent, irc or firefox, but i think this is the visual studio build, so i think that as soon as i click, it's gonna freeze.
oh, no, ok, alright. firefox 23, install, and this is actually going on mozilla.org and downloading the firefox installer, which hopefully will... this usually loads faster, but you're all probably on wi-fi now. alright, so we'll let that run in background, actually i'm going to get an older version, so that i don't make you wait forever. firefox 2, there we go. is that one smaller? still big,
let's try irc. there, that's better. and this is 100% mirc from mirc.com, it's not customized, it's not a special build for reactos, it's the actual mirc installer. almost done... let's hope it works, otherwise you're gonna see a reactos blue screen of death. it works just like a windows progress bar... and let's see if
we have the setup. yep, there it is. next, i agree, sure, next, next, superfast, and let's launch it, looks good and i could go and connect on irc server now. so this is unmodified windows app running on reactos and this actually works fairly long. i can't actually see that there's, like, i don't see any visible rendering issues, anything like that. so, that's when things go really well. when things don't go well,
then you just get a blue screen of death or things get rendered all over the place or things just look really, really, really bad. sound works as well, so we have like a... let's see... accessories, nope... multimedia player, and i can go play the logon wavefile. where's... the thing... there we go.
so, everything echoes on reactos. //so, everything else is now reactos. you wanna see a system properties, so it's- one of the things we try to do is how we have a very similar experience to how windows machine work. because a lot of companies - one of the reasons they don't wanna use linux is - they're afraid to move to something else cause if something goes wrong, they have to someone train that knows how to fix that on linux. or we have a device manager that looks just like on windows, when your properties looks just like on windows
and you know that on windows you can right-click and disable the device to get rid of it, you can do it in reactos as well. so, we're really trying to make sure that the interface and everything else works and looks like it should on windows. let me show you how you can debug this thing, so i'll actually shut- turn off for a second and launch windbg, the windows debugger, tell it to connect over a named pipe, start reactos
in debug mode and i'm gonna scroll up, cause this is quite funny. this is windbg thinking that it just connected to windows server 2003! and that it can't find the symbol files on the microsoft server. so this is just basically debug output from i'm gonna give it the right symbol path to where i've actually got the symbols built... and now it took the symbols and who ever used windbg, for those of you that took the training, for example, at google, "!process 0 0" is how it's going to give me all the
running processes by going into some very deep kernel-specific structures that only the windows kernel has. and it just gave me all the processes like it would on a real windows machine. i can do "!process 0 1" and see all the information about all the processes. i can see the current stack and the actual c files that correspond to these functions. which is as close as you're ever gonna get to nt kernel code
without working for microsoft. and i can step - if i had the sources checked out i could actually step - through kernel code with the source debugger, put a breakpoint, see variables, and i can still do it in assembly, but i don't have the sourcefile, the sources checked out on this things yet. so i can press go, go back. so, i'm honestly, really surprised that a super internal microsoft debugging tool can be fooled into working on something that's not really windows. so, last thing to show you is
how does this thing install, because we wanted to make sure even the installer looks like windows - windows xp. so if you've never installed windows xp probably don't remember this, but this is how it used to look like. yeah, that's right. and the only difference from xp is that this is how fast this thing installs. yeah, there we go. bam! thank you. and what happens when you reboot windows after going through text setup - you get the whole 2nd stage setup where all your devices are installed,
so now if i boot it up it's gonna boot up and register all the com components, asking for my username, my password, it's not going to ask you for your license key! name... there you go... next, next, next. it actually asks you to install mozilla right now so can actually have a browser. i'll say "no" and now the 2nd time i reboot i'm gonna get the desktop and back to where we started. so it's actually surprising that, honestly, this is probably 10% of whatever thing windows has. that's
why it builds so fast, links so fast, boots up so fast. but you can still run mirc without a single problem. that tells you how much bloat there is in windows, if all you wanna do is run bittorrent, irc and use a web browser. i even don't need all that. so, this is what happens to our beautiful add device wizard when you compile it with visual studio. with gcc it looks perfect, like, you'd have an actual text here. so something is different, we have a bug that's not
exposed in gcc, that is exposed in visual studio. and that's a really kinda awesome stuff you find out when you use multiple compilers. so i could demo this all day, bu i have better things to do. so, i'll open it up... to questions! and i can pass this around, i guess, or you can yell really, really loudly, or... you'll pass it on, there we go. mic lady: all right, questions? audience: hi there,
when you actually started your presentation, you actually stated there was a lot of legacy code in windows and i guess the best example i can think of right now is directshow which was being too long around and being like it's deprecated since windows vista but it's still there in windows 8. what is your best example when you are working on reactos of "oh god, that's still there?" alex: umm, does this work? someone? ok. so,
back in 1991, ibm was working on something they called abios - advanced bios, which is a 32-bit bios. and windows had a partnership with ibm and they went ahead to implement support in the kernel for abios. that never shipped. but we actually implemented it becase we didn't know if this mattered or not, so we did all the work to implement it and then somewhere around two years ago someone from google mountain view
found a horrible exploitable bug in the kernel by leveraging that codepath that noone had used in 12 years. that's when "oh god, we actually implemented that" and got rid of it. so i think that's one thing you find. another favorite of mine is that 386 had a bug in it that you could workaround. so most oss would detect what kind of 386 processor you have and then workaround the bug if needed. windows have that code too. now, around windows 2000 they stopped supporting 386
and around windows xp they dropped everything but the pentium. windows 8, today, still detects if you have that bug on 386, fixes the bug and then, about 15 microseconds later, crashes because you're running on 386. so, good question. audience: ok, so, first of all. since you're using wine, does that mean that to certain extent, steam can start running and then crash?
alex: i have not tried it, you should try that. audience: the follow-up question was - alex: i know that some 3d games actually work fairly well, like unreal tournament 2003 works, so, maybe steam works too. audience: ok, so the follow-up question would have been - would it run crysis? alex: not crysis, no. but, simcity 2000, yes. in fact, if you go to reactos.org/screenshots there's actually some screenshots and i'll open them up while i'm asked another question, i guess. audience: any other product that work on reactos?
like photoshop? alex: photoshop, that's a good question. i know that gimp works, i'm pretty sure older versions of photoshop work, i mean, so actually, one thing we have on our website is the app database, so... we're migrating the website, but on the old website we had an app compat database and you're asking about photoshop...
audience: any other product is gonna be fine. alex: yeah. i mean, there's a lot of products i can think about that work. audience: and any plan for running reactos on quantum computers? a alex: when you buy me a quantum computer to try on, i will let you know. audience: i actually built a mini quantum computer at home and i run gecko, firefox browser, that's only gecko and nothing else on the quantum computer, so... alex: you should take- download our source code and try to build it. and we actually have an arm port, for example, that some people are trying to get to work
on raspberry pi, so try it out. any other questions? to the back? audience: so you talked about how you have a lot of, like, different ways of testing things which you normally probably couldn't do in a regular os, things like, you know, dropping in a dll from windows. do you automate any of that or is that all using something like delta debugging or something or do you- is it just
manual? alex: the... specifically, everything is automated, normally, except the dropping the dlls. so the dropping the dlls is something we literally just discovered two weeks ago, that "wow! this works." so we're doing it manually, but we're actually in a process of setting up a build server that will- that basically has alongside of it all the windows dlls and we can tell it through... someone's building a web interface, so we can say "i want this dll" and then it'll install reactos, copy the dll and then run the tests, so we're working on automating it
because we think it's a really great thing. something i didn't show in the demo, by the way, is we actually support xp themes, so if you take your luna.msstyles or any custom xp theme, you can theme reactos. the theming dll actually works. audience: i have a question, how long does it take for someone to get himself or herself used to all the standards and the parameters, the structure of the os that you have there? because i assume there
are some documentations that people can use. alex: yeah, so, if you're not familiar with windows at all, it's probably gonna be a little bit longer. but if you're familiar with windows, you're gonna see reactos/dll in the slash all the dlls and you can go "cool, kernel32.dll" and here's all files, so reactos/kernel and if you know a little bit about what's in the kernel, you'll be able to see that. so, if all you're trying to do is work on an app, there's like
/applications, so it's... the stuff is pretty well ordered. now turns you're actually checking out your first build, building it, we have a wiki page that pretty much tells you from a to z how to check out, how to build, how to test, how to do all that. people are really friendly, like, these folks really wanna help out and you can come on irc, send a mailing list, so honestly, if you've done some windows development, even if it's been just a visual basic app, probably a week or two and you can
probably start contributing actual code. at the beggining your patches will on- will be as patches on jira, on the bug tracker, and then we'll review them for you, put them in code review, tell you what you did wrong, how you can do better, check them in and then after you submit a few patches and you're comfortable and we're comfortable, then you'll get a developer account and you can start checking in code like everyone else. i've got msn messenger screenshots here. so, noone really uses that anymore, but it still works.
the .net dll from wine actually works, so anything that uses passport should work out of the box as well. other questions? audience: so i was trying to think of the most awful part of windows that i could think of. i guess i came up with dcom. so i'm wondering how fully you've implemented dcom? alex: i'll answer that. the most important game in the world works on reactos. so, dcom is all from wine. so as good as wine implements dcom,
reactos implements dcom. except dcom uses named pipes and named pipes is our implementation and we actually just rewrote named pipes because of some bugs. so, dcom uses rpc which is named pipes. so, it should work as good as wine's dcom works and since wine runs office which is heavily dcom focused, i think fairly well. audience: i was wondering if you knew at the time of pclinuxos which was doing similar thing to reactos
and if you know about the differences between them? they were, if i rememeber well, they were using their 3d libraries pretty early? alex: i know some projects that took- that would take linux and run windows libraries natively on that, so a little bit more than wine is doing but not so reactos. so i don't know about all of them specifically, i don't think i've heard about this one. (someone from audience asks a question without microphone) alex: yeah, some people ask us about that and i know that if you take certain portable apps
and you can just run them on reactos as well, so some people are basically trying to make basically trying to make portable reactos where you take a linux install and you have a vm with reactos with the portable app and you just drag and drop and now this windows app works. it's just almost pretty much with game developers doing with mac versions, so it's a portable app version of wine, so we were looking at some of that as well, but we're not quite there yet. (someone from audience asks a question without microphone) yep, exactly. now, diablo 2 also works
in reactos. audience: hello, my name is olaf. i'm wondering if it's self-hosting, so do you run visual studio? alex: you cannot run visual studio on reactos yet and probably when we do that, we'll be done, cause that's one of most massive, complex pieces of windows code ever. but gcc does run so you can actually build- you can run cmake and ninja and gcc and you can build reactos in reactos if the memory manager works
out and if the file manager is happy. so, if you've got a really good disk and you'd give it a lot of ram, it should work out. it's not something i'd recommend yet, but it works. audience: yeah, another thing, you know, related to that because, i mean, not needing windows to build something would be great. how about something like testing websites and stuff like that, i mean, there's a lot people trying to do that or- alex: well, i mean, in terms of websites we use either gecko or firefox or opera. chrome
doesn't run yet, so- audience: internet explorer. alex: yeah, internet explorer is so deeply tied to windows that it will never work on reactos probably and we don't wanna reimplement. we don't wanna reimplement a browser when there's already really good browsers out there. so what we actually have is we have gecko that we load in an ie-compatible frame so the ie com object - we render that through gecko which is what ie does, but in terms of website testing i'm not sure it's gonna help much.
cause you're gonna see the same stuff opera sees on any other os plus all the rendering bugs reactos has. so once you really wanna fix your website to render correctly on reactos, thank you - probably not worth it. mic lady: he will take two questions. audience: what kind of language is used to automate the test cases? alex: most of reactos is written in c, i actually have a little chart here to show you. so this is like our line of code or something. most of it is c ...so, that's not code metrics...
we have a little bit of c++ and most of the web stuff is in php, javascript and so on and so forth. so, we follow the same language that windows wrote, which is mostly, 99%, c. last stuff you'll see is c++, though. audience: so, the tester platform is open for public? alex: the test platform uses... oops, this is not what i want to show, just realized... the test platform is, like, the testman is all php, autohotkey is an opensource tool that someone else wrote, what we really customize is sysreg - sysreg2 - and those are
in public, so if you have svn, reactos/tools/regression, you'll see all the tools here. so the whole website is in a git repo and all the tools we build are in a repo as well. so not just the os is opensource, but everything we've built to support the os is also opensource. so yes, you can use our testing platform on something else you might be working on if you want to. audience: so everybody can build their own testing platform, right? lex: yup, ok. - thanks. - sure. and i'll hang out, so those of you who are not asking questions, you can ask me later,
you know, after this is done, no problem. audience: alright, so you said that the project aims for compatibility with windows xp, but what if for example in windows 7 there's some function that's been added to user32 or something like that? will you also implement that function or will you just refrain from doing it because windows xp didn't have it? alex: so, yeah, support for windows 7 and other functions has kind of really been on top of my head, on the top of our head. the reason i- my first answer like two years ago would have been "no.
we would stick to xp." but what's happening is that we are starting to see some applications that have dependencies. right, so the reason i said "no, xp only" is because we're chasing a moving target. it's very hard to know it's my bug, because i have implemented... there's, actually, there's a bug in xp dll that if you fix the bug, obviously the kernel won't work. but, in... sorry, there's a bug in the kernel and if you fix it, then the dll won't work. but if you reverse engineer the windows 7 dll, the bug is no longer in that dll
because they also fixed it in the kernel. so i had someone argue with me saying "no, we should get rid of this bug in this dll cause, look, windows doesn't have it" i'm like "no, you're looking in the windows 7 dll, 2003 dll fixed it" and he's asking "well how come the bug is not there?" - because they also fixed it on the other side. so, my worry was, if we're doing half-and-half, maybe the windows 7 function works that way because this other windows 7 dll works this way and windows 7 kernel does it this way. so it's very hard to claim that
i'm compatible if you're mixing and mathing. now, that being said, we have done exceptions. so it's like there's a windows 7 function in user32 whose sole job it is to like calculate the matrix of a rectangle. well, that doesn't need anything else than, like, math and we can implement that. so, stuff like that, we've done on the side, but another thing about that is, sometimes if you export or implement or claim to have a windows 7 function, some apps will see that you're exporting it and will use that to say "oh, you must be windows 7, i'm gonna do all these other things too!"
so we're actually trying to build a system where you can at install time say "do you want to have xp behaviour or do you wanna have some of the windows 7 functions? and, you know, your mileage may vary". so, we're kinda starting to be on the fence about that, first in user mode stuff. in the kernel, though, we're gonna stick to 2003, once we're fairly sure it's working, probably actually go to 7 or 8.1. so, that's the long answer. so thanks a lot. we also have a coverity scan. thank you!
shibl: thank you alex, this is a very, very interesting talk and amazing projects, thank you. the next thing you need to do is to get chrome to work. alex: yes, and in coverity here is 4,000 bugs that coverity found, you can get started on them right now! thank you. shibl: thank you and thank you everybody for coming.
0 comments:
Post a Comment