Good day to all! 
My name is Stanislav, and I love writing code. This is my first article on Habré, to write which I was inspired by several factors:
Let me introduce you to the culprits of this celebration (fixed bug that prevented the launch of Git in ReactOS) - French developer Hermès Bélusca-Maïto (hereinafter simply Hermes, with the nickname hbelusca ), and I myself (with the nickname x86corez ).
The story begins with the following messages from the ReactOS IRC channel:
Jun 03 18:52:56 <hbelusca> Anybody want to work on some small problem? If so, can someone figure out why this problem https://jira.reactos.org/browse/CORE-12931 happens on ReactOS? :D Jun 03 18:53:13 <hbelusca> That would help having a good ROS self-hosting system with git support. Jun 03 18:53:34 <hbelusca> (the git assertion part only).
Debriefing
Since at the moment ReactOS is aimed at compatibility with Windows Server 2003, Git version 2.10.0 was chosen as the guinea pig, the latter with support for Windows XP and 2003.
Testing was done on the ReactOS command line, the external symptoms of the problem were rather ambiguous. For example, when you run git without specifying additional arguments, it output reference information to the console without any problems. But it was worth trying git clone or at least git --version , in most cases the console did not display anything at all, or occasionally showed a broken message with assertion:
git.exe clone -v "https://github.com/minoca/os.git" "C:\Documents and Settings\Administrator\Bureau\minocaos" A ssertionfailed ! P rogram : C : \ P rogram F iles \ G it \ mingw 3 2 \ bin \ git . exe F ile : exec _ cmd . c , L ine 2 3 E xpression : argv 0 _ path This application has requested the Runtime to terminate in an unusual way. Please contact the application's support team for more information.
It was very helpful to the investigation of the problem that the git client was open source, and finding the line on which the exception occurred was not too difficult: https://github.com/git-for-windows/git/blob/4cde6287b84b8f4c5ccb4062617851a2f3d7fc78/exe c # L23
char *system_path(const char *path) { #ifdef RUNTIME_PREFIX assert(argv0_path); // assert(is_absolute_path(argv0_path)); #endif strbuf_addf(&d, "%s/%s", prefix, path); return strbuf_detach(&d, NULL); }
And the value of the variable argv0_path set by this piece of code:
const char *git_extract_argv0_path(const char *argv0) { const char *slash; if (!argv0 || !*argv0) return NULL; slash = find_last_dir_sep(argv0); if (slash) { argv0_path = xstrndup(argv0, slash - argv0); return slash + 1; } return argv0; }
After clarifying these details, I unsubscribed on the IRC channel:
Jun 03 19:04:36 <x86corez> hbelusca: https://github.com/git-for-windows/git/blob/4cde6287b84b8f4c5ccb4062617851a2f3d7fc78/exec_cmd.c#L23 Jun 03 19:04:41 <x86corez> assertion is here Jun 03 19:04:57 <hbelusca> yes I know, I've seen the code yesterday. The question is why it's FALSE on ROS but TRUE on Windows. Jun 03 19:06:02 <x86corez> argv0_path = xstrndup(argv0, slash - argv0); Jun 03 19:06:22 <x86corez> xstrndup returns NULL %-) Jun 03 19:06:44 <hbelusca> ok, so what's the values of argv0 and slash on windows vs. on ROS? :P Jun 03 19:08:48 <x86corez> good question!
The variable name as if hints that the value was obtained from argv[0] - usually in the zero index of this array contains a line with the name of the command, which was called the current program. But in the future everything was not so obvious ...
Jun 03 20:15:21 <x86corez> hbelusca: surprise... git uses its own xstrndup implementation Jun 03 20:15:35 <x86corez> so I can't simply hook it xD Jun 03 20:15:56 <hbelusca> well, with such a name "xstrndup" it's not surprising it's its own implementation Jun 03 20:16:04 <x86corez> probably I would need an user-mode debugger... like OllyDbg Jun 03 20:16:09 <hbelusca> that's everything but standardized function. Jun 03 20:16:24 <hbelusca> x86corez: ollydbg should work on ROS. Jun 03 20:16:30 <mjansen> what are you breaking today? Jun 03 20:16:44 <x86corez> mjansen: https://jira.reactos.org/browse/CORE-12931 Jun 03 20:16:51 <hbelusca> (of course if you also are able to compile that git with symbols and all the stuff, it would be very nice)
Transition to action
After that, I decided to build git from source so that it would be easier to print the values of the variables of interest directly to the console on the fly. I chose this step-by-step manual , and I chose this thread to build a compatible version: https://github.com/git-for-windows/git/tree/shears/v2.10.0-rc2
The setup of the mingw32 toolchain passed without problems, and I started the build. But in such step-by-step manuals there are often undocumented pitfalls, and I immediately came across one of them:
Through trial and error, as well as using tips from the audience (IRC channel), all the compilation problems were solved. If someone wants to repeat my path, I share a ready-made diff for a successful build: https://pastebin.com/ZiA9MaKt
In order to eliminate the influence of many functions during initialization, I decided to display several debugging messages right at the beginning of the main() function, which in the case of git is located in the common-main.c file
int main(int argc, const char **argv) { //DebugBreak(); printf("sanitize_stdfds(); 1\n"); sanitize_stdfds(); printf("git_setup_gettext(); 1\n"); git_setup_gettext(); printf("sanitize_stdfds(); 2\n"); sanitize_stdfds(); printf("git_setup_gettext(); 2\n"); git_setup_gettext(); printf("before argv[0] = %s\n", argv[0]); argv[0] = git_extract_argv0_path(argv[0]); printf("after argv[0] = %s\n", argv[0]); restore_sigpipe_to_default(); printf("restore_sigpipe_to_default(); done\n"); return cmd_main(argc, argv); }
The conclusion was the following:
C:\>git --version sanitize_stdfds(); 1 git_setup_gettext(); 1 sanitize_stdfds(); 2 git_setup_gettext(); 2 before argv[0] = git after argv[0] = git restore_sigpipe_to_default(); done A ssertionfailed ! ( , )
It would seem that everything is fine, argv[0] and this should be. The idea came to drive git inside the debugger, for example OllyDbg, but something went wrong ...
Jun 04 01:54:46 <sanchaez> now please try gdb/ollydbg in ROS Jun 04 01:58:11 <sanchaez> you have gdb in RosBE Jun 04 01:58:20 <sanchaez> just in case :p Jun 04 01:59:45 <x86corez> ollydbg says "nope" with MEMORY_MANAGEMENT bsod Jun 04 02:00:07 <x86corez> !bc 0x0000001A Jun 04 02:00:08 <hTechBot> KeBugCheck( MEMORY_MANAGEMENT ); Jun 04 02:00:13 <hbelusca> :/ Jun 04 02:00:49 <sanchaez> welp Jun 04 02:00:56 <sanchaez> you only have one option now :D
And here sanchaez suggested a great idea that shed light on a lot!
There were no exceptions, and git successfully printed its version number.
Jun 04 02:23:40 <x86corez> it prints! Jun 04 02:23:44 <x86corez> but only in gdb Jun 04 02:23:53 <hbelusca> oh Jun 04 02:24:00 <hbelusca> C:\git/git.exe Jun 04 02:24:13 <hbelusca> I wonder whether it's the same in windows, or not.
The case moved from a dead end, and I decided to try differently to run git on the command line, and did not lose!
The problem was obviously that git was expecting the full path on the command line. Then I decided to check what data will be displayed in Windows. The results surprised me a little.
The variable argv[0] for some reason had the full path to the application.
Jun 05 23:01:44 <hbelusca> x86corez: can you try to run git also by not using cmd.exe? Jun 05 23:02:05 <hbelusca> (to exclude the possibility it's cmd that doesn't call Createprocess with a complete path) Jun 05 23:02:09 <hbelusca> while I think it should... Jun 05 23:02:30 <x86corez> not using cmd... moment Jun 05 23:02:55 <hbelusca> x86corez: alternatively, on windows, try starting git using our own cmd.exe :)
Hermes offered to check, maybe the problem is somewhere in the ReactOS command line ...
But it was not so. The option with a shell cmd disappears.
Jun 05 23:04:38 <x86corez> ROS cmd is not guilty Jun 05 23:07:57 <hbelusca> If there was a possibility to consult the received path, before looking at the contents of argvs... ? Jun 05 23:08:30 <x86corez> dump contents of actual command line? Jun 05 23:08:39 <hbelusca> yeah Jun 05 23:09:39 <hbelusca> The thing you retrieve using GetCommandLineW Jun 05 23:10:03 <hbelusca> (which is, after simplifications, basically : NtCurrentPeb()->ProcessParameters->CommandLine ) Jun 05 23:10:59 <hbelusca> Also I was thinking it could be a side-effect of having (or not having) git path into the env-vars.... Jun 05 23:12:17 <x86corez> hbelusca, command line is "git --version" Jun 05 23:12:34 <hbelusca> Always? Jun 05 23:12:39 <x86corez> Yes, even on Windows Jun 05 23:15:13 <hbelusca> ok but then it would be nice if these different results are at least the same on Windows and on ROS, so that we can 100% exclude problems outside of msvcrt.
Now it only remained to test the msvcrt.dll library from ReactOS for Windows. I tried to put the file in the same directory where git.exe was located, but this did not help. Mark advised a way with the .local file:
Jun 05 22:59:01 <mjansen> x86corez: add .local file next to msvcrt.dll ;) Jun 05 22:59:47 <mjansen> exename.exe.local Jun 05 23:00:17 <x86corez> just an empty file? Jun 05 23:00:21 <mjansen> yea Jun 05 23:00:49 <hbelusca> mjansen: do we support these .local files? Jun 05 23:00:52 <mjansen> we dont Jun 05 23:00:54 <mjansen> windows does Jun 05 23:15:48 <x86corez> moment... I'll try with .local Jun 05 23:18:43 <x86corez> mjansen: I've created git.exe.local but it still doesn't load msvcrt.dll in this directory
But for some reason this option did not work either. Perhaps it was the fact that I conducted all the experiments on the server version of Windows Server 2008 R2.
The latest idea was proposed by Hermes:
Jun 05 23:19:28 <hbelusca> last solution: patch "msvcrt" name within git and perhaps other mingwe dlls ^^ Jun 05 23:20:12 <x86corez> good idea about patching!
With WinHex, I replaced all occurrences of the msvcrt substring in the msvcrt file with msvcrd , and renamed msvcrt.dll from ReactOS accordingly, and here’s what happened:
Jun 05 23:23:29 <x86corez> Yes! guilty is msvcrt :) Jun 05 23:25:37 <hbelusca> ah, so as soon as git uses our msvcrt we get the problem on windows. Jun 05 23:25:38 <x86corez> hbelusca, mjansen, https://image.prntscr.com/image/FoOWnrQ4SOGMD-66DLW16Q.png Jun 05 23:25:58 <hbelusca> aha and it asserts <3 Jun 05 23:26:03 <hbelusca> (it shows the assertion now)
Now in Windows we got the same error message! This means that the source of the problem is the implementation of one of the msvcrt functions from ReactOS.
You can also note that under Windows the exception text is displayed correctly.
Jun 05 23:26:13 <x86corez> but it prints text and correctly. Jun 05 23:26:20 <hbelusca> oh Jun 05 23:26:33 <x86corez> and on ROS it doesn't print in most cases xD Jun 05 23:26:38 <hbelusca> so also it excludes another hypothesis, namely that it could have been a bug in our msvcrt/crt Jun 05 23:26:56 <hbelusca> So possibly a strange bug in our console
It only remained to find out which function in msvcrt provides the full path to the executable file of the running application. A little googling, I assumed that the matter is in the _pgmptr function.
Jun 06 00:07:43 <x86corez> https://msdn.microsoft.com/en-us/library/tza1y5f7.aspx Jun 06 00:07:57 <x86corez> When a program is run from the command interpreter (Cmd.exe), _pgmptr is automatically initialized to the full path of the executable file. Jun 06 00:08:01 <x86corez> this ^^) Jun 06 00:08:50 <hbelusca> That's what GetModuleFileName does. Jun 06 00:09:04 <x86corez> yeah Jun 06 00:10:30 <hbelusca> Of course in ROS msvcrt we don't do this, but instead we initialize pgmptr to what argv[0] could be. Jun 06 00:11:08 <hbelusca> That's one thing. Jun 06 00:11:34 <hbelusca> The other thing is that nowhere it appears (in MS CRT from VS, or in wine) that argv is initialized using pgmptr. Jun 06 00:13:33 <x86corez> hbelusca, I've checked argv[0] in some ROS command line tools, running them in Windows Jun 06 00:13:56 <x86corez> they all interpret argv[0] as command line, not full path Jun 06 00:14:04 <x86corez> so... I think it's git specific behaviour Jun 06 00:14:16 <x86corez> or specific mingw compiler settings Jun 06 00:28:12 <hbelusca> x86corez: I'm making a patch for our msvcrt, would be nice if you could test it :) Jun 06 00:28:21 <x86corez> I'll test it
Hermes sent a link to the patch, I manually applied it and rebuilt the system, after which everything magically worked as it should!
Jun 06 00:34:26 <x86corez> hbelusca, IT WORKS! Jun 06 00:35:10 <hbelusca> LOL Jun 06 00:35:18 <hbelusca> So it seems that something uses pgmptr to rebuild an argv. Jun 06 00:35:52 <x86corez> I've even able to clone :) Jun 06 00:36:19 <hbelusca> \o/ Jun 06 00:36:21 <gigaherz> 2.10.0-rc2? not the release? Jun 06 00:36:24 <hbelusca> ok I'm gonna commit that stuff. Jun 06 00:36:43 <hbelusca> x86corez: gonna have ROS self-hosting <33 Jun 06 00:36:48 <x86corez> yeah! Jun 06 00:37:01 <x86corez> gigaherz: I've built that from sources Jun 06 00:37:37 <gigaherz> oh, for testing this bug? o_O Jun 06 00:37:50 <sanchaez> yes, you missed the fun :p Jun 06 00:39:46 <x86corez> git 2.10.0-windows.1 (release) works too! Jun 06 00:39:54 <encoded> commit!!!
Afterword
Thus, another bug that indirectly impedes the self-assembly of ReactOS within ReactOS, has been fixed thanks to collective efforts. A funny coincidence is the fact that shortly before this, another bug was fixed in the same msvcrt (namely, in the qsort function), which did not allow to assemble USB drivers in ReactOS.
I participate in the development of many projects written in different programming languages, both closed and open source. I have been cooperating with the ReactOS project since 2014, but I began to actively help and write code only in 2017. It is especially interesting to work in this area, because it is an entire operating system! It feels the huge scale of the result, in which efforts were invested, as well as the pleasant feeling that one bug was less! :)
Someone probably wonders why I was helping ReactOS, and not Linux, for example. Historically, in most cases I write programs for Windows, and my favorite programming language is Delphi. Perhaps that is why the architecture of Windows NT along with the Win32 API is very interesting to me, and the project of the free operating system ReactOS embodies a long-held dream into reality - in practice it allows you to find out how it all works from the inside.
I hope you were interested in reading my first article here. I look forward to your comments!
Links