Building with Make, Regex and F4

[Lozminda]

Hi

Firstly an apology to the team at ST3 (or ST as now is) @wbond, @bschaaf (not sure if you’re a dev, you’ve answered a lot of my questions, usually with have you tried a fresh install ) @OdatNurd et al. Recently I thought I’d try out some other IDE’s (I know ST3 isn’t an IDE) but OMG what a palarva. Netbeans, CodeBlocks, a bit ugly and could I get em to work, jeesh. Basically I’ve come running back to ST and am thinking of learning python and writing some xtra C++ functionality rather than try and get those beasts to work. I’m beginning to see why folks love this product, I didn’t see it at first, now I’m beginning to understand. So I apologise unreservedly for all my moaning during set up. (I’m a Brit, it’s what we do best, moan) (usually about the weather, if it’s not that our poota set up)

Now that’s out of the way, some moaning about ST3 ! (ha ha see what I did there)

Thanks for getting this far, to my question:

Am using a makefile to build C++ projects, and this alters the ouput that appears so that the regex in my build system doesn’t work and therefore F4 doesn’t work. I was wondering how to change the regex so that maybe it ignores the first few lines? And only passes the error messages on ?
Here’s some stuff that might help:

As you can see four lines of clang++3.8 -std=c++14 -stdlib=libc++ etc comes before the error messages.

ST Build file:

{
  "cmd" : ["make --file=makeSDL2with02.mk -B TARGET=\"$file_base_name\""],
  "selector" : "source.c",
  "shell": true,
  "file_regex":"^(|..[^:]*):([0-9]*):?([0-9]*)?:? (.*)$",
  "working_dir" : "$file_path"
}

As far as I understand it (by dropping it into this nifty regexer) it’s looking for the error pattern : : : with some line numbers ?
How do I alter the regex so that it ignores anyhthing that isn’t that pattern (if indeed that’s what it is) ? i.e. just skips through the build output until it gets to some errors and then F4? would work?
Am I making sense ? (before anyone says “learn regex”, please don’t, it’ll take me weeks to get good enough to actually come up with a pattern that does the right thing plus I’m not sure of the ST3 functionality in this case anyways)

Thanks y’all, hope everyones Okay during these crazy times.

Lozminda

[bschaaf]

How do I alter the regex so that it ignores anyhthing that isn’t that pattern (if indeed that’s what it is) ?

If the regex doesn’t match, it’s ignored. This is already what happens. You don’t need to do anything extra to skip lines that don’t match. The regex works just fine for me, perhaps it’s not finding the file from the path in the error?

[Lozminda]

I’m guessing it isn’t the path, because it does print out the error messages and does some highlighting.

The directory is correct, and the path that refers to the linux variable $PATH where it looks for executables (I believe)
Also the build system I included in the OP is a modification of other build files, which do work.
More annoyingly sometimes it does work, sometimes I get the error messages (in the red banners) (but no F4 functionality). Am studying atm, but I’ll try and make a note of when I’m getting the various levels of correct and partial operation. This has only happened since making files. Compiling and linking source files; F4 still works. Any other info you need to make a diagnosis?

L

Ps

perhaps it’s not finding the file from the path in the error?

Sorry which file (the error file, if there is one, executable, source or some other), and which path (dir, file_path, $PATH ) (and possibly) which error ?

PPs I’ve had a look at this but maybe it’s talking about ST2 ?

[bschaaf]

Sorry which file (the error file, if there is one, executable, source or some other), and which path (dir, file_path, $PATH ) (and possibly) which error ?

PosterSurface.cpp, the file from the error message. If it can’t be found from the working directory then it won’t be opened.

[Lozminda]

It’s there…

[OdatNurd]

Does your Makefile change directories as it’s building?

[Lozminda]

@OdatNurd in answer to your question No. One thing that does cause a problem, that I’ve identified, is if I use the ‘-v’ flag (for verbose compilation (and linking ) output) F4 doesn’t work ie with the ‘-v’ flag No F4 functionality.

So to say tentitively (I’ve no idea how to spell that) it might have some thing to do with the output from what ever’s doing the compilation. Anyway another piece to the puzzle…

[Lozminda]

Ps @seanbirdsell thanks for the like, does that mean you’re having the same problem too or you just liked my grovelling apology

[seanbirdsell]

Merely applauding a great break down of a hard error to diagnose.

[seanbirdsell]

Also, mom’s English so I understand the whinging.

[OdatNurd]

The regex items provided in the build need to be able to match the error lines, which tells Sublime what filename, line, column and error/warning/info message is associated so that it can navigate.

Files that are absolute filenames will open directly but anything that’s a relative filename (i.e. somefile.txt instead of c:\some\long\path\somefile.txt) will be opened from a folder determined from the working_dir.

Thus anything that changes the output can stop it from matching, and anything that provides a filename but doesn’t (or can’t) provide a working_dir to know where the file exists can’t be opened.

[Lozminda]

@OdatNurd, thanks once again. I’m gonna have to let that stew a bit as on first reading it’s gone a bit over my head…I shall ponder…ATB

[Lozminda]

Some thoughts (if I’e understood you correctly):

That means when I compile with the verbose option, the compiler/linker is outputting messages to a different working directory depending on whether the verbose flag is set ?

I’ve got to find out where make (and maybe some other processes) send their output too (???) and change the working dir accordingly (???)

Or am I totally off piste here ?

L

[Lozminda]

Thanks, nice to be appreciated ! (even if it is for moaning )

[OdatNurd]

That means when I compile with the verbose option, the compiler/linker is outputting messages to a different working directory depending on whether the verbose flag is set ?

Here’s an example:

{
	"shell_cmd": "g++ \"${file}\" -o \"${file_path}/${file_base_name}\"",
	"file_regex": "^(..[^:]*):([0-9]+):?([0-9]+)?:? (.*)$",
	"working_dir": "${file_path}",
	"selector": "source.c++",

	"variants":
	[
		{
			"name": "Run",
			"shell_cmd": "g++ \"${file}\" -o \"${file_path}/${file_base_name}\" && \"${file_path}/${file_base_name}\""
		}
	]
}

The file_regex here is what allows Sublime to detect the file, line, column and error message. The only things that are going to match are lines that match this regex. Anything that doesn’t match is entirely ignored.

So, vitally important step #1: if you want to navigate between errors, this regex needs to be able to match them. If passing extra command line arguments to the tool you’re running changes the way the error messages are presented, then you also need to change this, or it’s not going to match, and you’re not going to be able to navigate.

Now, presume an error message like this:

/home/tmartin/test.c:8:13: error: expected ‘;’ before ‘}’ token

The name of the file is absolute; it is always possible to know exactly what file on my hard drive is the one that triggered the error.

On the flip side:

test.c:8:13: error: expected ‘;’ before ‘}’ token

Ok, a file named test.c has an error. Which one? There can be arbitrarily many of them. I could be some sort of freak with a test.c in every folder on my entire hard drive. Which one is it referring to? Sublime can’t know, it’s not the thing that generated the error message, the external tool is. This is the inherent problem with relative paths. They’re relative to some known location, but what is that location?

	"working_dir": "${file_path}",

This part of the build system says “while you are executing this tool, as a first step please change the current working directory to be the path in which the currently active file is stored”.

Given this, Sublime can presuppose that if it has a relative filename, it has to be relative to something, and the only information it has is the directory that was currently active while the tool was running. So, it will stick the relative filename onto this working directory path and see if that opens the file.

Vitally important step #2: If you don’t set the working directory, OR the tool that you’re executing changes it, AND the error message is a relative filename, there’s no way to find and open the file.

The import of the my original statement is something like this:

(master *%>) tmartin:dart:~/local/src/dragonWarrior> make
make[1]: Entering directory '/home/tmartin/local/src/dragonWarrior/external'
make[2]: Entering directory '/home/tmartin/local/src/dragonWarrior/external/lua'
make[2]: Nothing to be done for 'install'.
make[2]: Leaving directory '/home/tmartin/local/src/dragonWarrior/external/lua'
make[2]: Entering directory '/home/tmartin/local/src/dragonWarrior/external/mersenne'
make[2]: Nothing to be done for 'install'.
make[2]: Leaving directory '/home/tmartin/local/src/dragonWarrior/external/mersenne'
make[2]: Entering directory '/home/tmartin/local/src/dragonWarrior/external/md5'
make[2]: Nothing to be done for 'install'.

This build is recursive; every time it goes into a folder, it’s changing what the currently active directory is. If your build does that, AND your tool outputs a relative filename like hello.c, then you’re screwed and you can’t open the file because Sublime has no way to know what file it could possibly be; the external tool changed information out from under it.

[Lozminda]

Thanks, A Fantastic Reply, it’s a bit late where I am, I’ll get into in the next day or two…Awesome., great work Mr Martin !

L

[Lozminda]

Some cross over from this post

I notice that you have this as you working dir:

	"working_dir": "${folder:${project_path:${file_path}}}",

Would that fix the issue I’m having in this post I wonder…

Ok I think I’ve understood (think being the word here)

I saved this project I’m testing in the sae folder as the source code.

Here’s output from building using clang and the build file:

As you can see F4 works. Notice the error format WLM007.cpp:37:4: error: blar

Here’s the build file:

	"cmd" : ["clang++-3.8 -std=c++14 -stdlib=libc++ -lSDL2 -lSDL2_image -lgmpxx -lgmp -O02 -pthread -w ${file_name} -o ${file_base_name}"],
"selector" : "source.c",
"shell": true,
"file_regex":"^(|..[^:]*):([0-9]*):?([0-9]*)?:? (.*)$",
"working_dir" : "$file_path"
}

So now building with the makefile:

The error format is the same as above, so I’m guessing that means your first point isn’t what’s happening here:

00000000000000000000000000000000000000000000000000000000000000000000

The name of the file is absolute;

to

So, it will stick the relative filename onto this working directory path and see if that opens the file.

00000000000000000000000 END QUOTE 000000000000000000000000000000000

Here’s the build file for the makefile:

{
  "cmd" : ["make --file=makeOdatNurd.mk -B TARGET=\"$file_base_name\""],
  "selector" : "source.c",
  "shell": true,
  "file_regex":"^(|..[^:]*):([0-9]*):?([0-9]*)?:? (.*)$",
  // "working_dir" : "$file_path"
  "working_dir": "${folder:${project_path:${file_path}}}",
  "syntax": "Packages/Makefile/Make Output.sublime-syntax"
}

The make file makeOhDatNurd (hope you don’t mind !) is in the same directory as the source code. I thought changing the working dir to the above might help. It’s given me some nice colouration…And I thought the syntax might help…

Surely if ST is displaying the error info ? There clearly something I’m missing. I’ve read your post several times, I believe I understand whats going on… Ha ha belief.

Cheers, Loz
Ps I changed

“working_dir”: “${folder:${project_path:${file_path}}}”

to (and I then get no error messages and F4 doesn’t work as opposed to error messages and F4 doesn’t work…)

“working_dir”: “${project_path:${file_path}}}”,

[Lozminda]

How d you get make to printout which directory it’s “going into” ? The makefile I’m using is in the same directory as the source code and all the object files too. But just in case.

Also there’s something else that makes me think it isn’t reason 2 you gave:
If I keep pressing F4 nothing happens in the panels where my code is until I get to the end and then it opens this funny dir tab with nothing in it…

[OdatNurd]

How d you get make to printout which directory it’s “going into” ?

My make is doing that automatically because my Makefile looks something like this:

install clean release::
    @cd external && $(MAKE) $@

Building with this Makefile tells make to go inside of the external folder and recursively call make again to do the build in there (and exernal also has a Makefile in it). Since make knows that it’s invoking itself again, it tells you when it’s entering and leaving folders like this so you can track what it’s doing.

For a build that’s not doing this, the most expedient thing would be to add pwd (or if you’re on windows, cd) as a command under the make rule, which should display the current folder while that rule is executing.

[Lozminda]

I’ve printed out the directory from the makefile, it’s the same as the source file and error messages, so it remains a mystery !
(I split my recipe with ; pwd)