Stack overflow while generating tags completion table in emacs

7
votes

I'm using GNU Emacs 23.3 on Windows. I work in a very large codebase for which I generate a TAGS file (using the etags binary supplied with Emacs). The TAGS file is quite large (usually hovers around 100MB). I rarely need to use any functionality beyond find-tag, but there are times when I wish I could do completion out of the TAGS table.

Calling complete-tag causes Emacs to make a completion table automatically. The process takes quite a bit of time, but my problem isn't in the amount of time it takes, but rather the fact that right at the end (around 100% completion), I get a stack overflow (sorry about the unprintable chars):

Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
  re-search-forward("^\\(\\([^]+[^-a-zA-Z0-9_+*$:]+\\)?\\([-a-zA-Z0-9_+*$?:]+\\)[^-a-zA-Z0-9_+*$?:]*\\)\\(\\([^\n]+\\)\\)?\\([0-9]+\\)?,\\([0-9]+\\)?\n" nil t)
  etags-tags-completion-table()
  byte-code(...)
  tags-completion-table()

Has anyone else run into this? Know of a way to work around it?

EDIT: Stack output after turning on debug-on-error

EDIT: Removed stack, since I now know what the failing entries look like:

^L
c:\path\to\some\header.h,0
^L
c:\path\to\some\otherheader.h,0

My tags file contains quite a few entries in this format. Looking at the headers involved, it's clear that they couldn't be correctly parsed by etags. This is fine, but I'm surprised that tags-completion-table doesn't account for this format in its regex. For reference, here's what a real entry looks like:

^L
c:\path\to\some\validheader.h,115
class CSomeClass ^?12,345
bool SomeMethod(^?CSomeClass::SomeMethod^A67,890
3
emacstags
I now know how to fix up my TAGS file after the fact so that tags completion generation succeeds. Feel free to post suggestions on how to prevent the bogus entries from being generated in the first place... - paulcam
I work with matlab files so I use etags with a bunch of custom regex'es, but I've found that "bad entries" in the TAGS table occur when I ask etags to parse a script where it doesn't find any interesting tags. It creates this sort of "null" entry that you've indicated, where it just returns path,0. Not sure how to suppress this behavior. Fortunately, it seems that these are always at the bottom of the TAGS file, so should be pretty easy to write a cleaner for it. - Daniel Kessler

3 Answers

3
votes

The regexp in question is used to match a tag entry inside the TAGS file. I guess that the error can occur if the file is incorrectly formatted (e.g. using non-native line-endings), or if an entry simply is really, really large. (An entry is typically a line or two, which should not be a problem for the regexp matcher.)

One way of tracking down the problem is go to the TAGS buffer and see where the point (cursor) is, after the error has occurred. Once you know which function it is, and you could live without tags for it, you could simply avoid generating TAGS entries for it.

If the problem is due to too complex entry, I would suggest that you should send bug report to the Emacs team.

1
votes

If you load the tags table (open the TAGS table with Emacs, then bury-buffer), try M-x dabbrev-expand (bound to M-/). If the present prefix is very common, you might end up running through many possible completions before reaching the desired one.

I don't use Windows, but on the Mac and Linux machines I use, I have not faced this issue.

0
votes

This looks like a bug in Emacs, see:

I have applied the suggested patch to etags-tags-completion-table (copied below in completeness for your convenience) and trapped an error case.

I'm triggering the error in an extremely long line of code (46,000 characters!). I presume somebody programmatically generated the line and pasted it into the source. A workaround could be to simply filter such lines at the ctag building or loading stage, just something that deletes "long" lines, whatever that may mean. Probably 500 characters is long enough!

I could also look at adding maximum sizes to my regexes in ctags, but that really isn't a general solution because many ctags patterns do not have such limits. 

(defun etags-tags-completion-table () ; Doc string? 
  (let ((table (make-vector 511 0)) 
        (progress-reporter 
         (make-progress-reporter 
          (format "Making tags completion table for %s..." buffer-file-name) 
          (point-min) (point-max)))) 
    (save-excursion 
      (goto-char (point-min)) 
      ;; This monster regexp matches an etags tag line. 
      ;;   \1 is the string to match; 
      ;;   \2 is not interesting; 
      ;;   \3 is the guessed tag name; XXX guess should be better eg DEFUN 
      ;;   \4 is not interesting; 
      ;;   \5 is the explicitly-specified tag name. 
      ;;   \6 is the line to start searching at; 
      ;;   \7 is the char to start searching at. 
      (condition-case err 
          (while (re-search-forward 
                  "^\\(\\([^\177]+[^-a-zA-Z0-9_+*$:\177]+\\)?\ 
\\([-a-zA-Z0-9_+*$?:]+\\)[^-a-zA-Z0-9_+*$?:\177]*\\)\177\ 
\\(\\([^\n\001]+\\)\001\\)?\\([0-9]+\\)?,\\([0-9]+\\)?\n" 
                  nil t) 
            (intern        (prog1 (if (match-beginning 5) 
                               ;; There is an explicit tag name. 
                               (buffer-substring (match-beginning 5) (match-end 5)) 
                             ;; No explicit tag name.  Best guess. 
                             (buffer-substring (match-beginning 3) (match-end 3))) 
                      (progress-reporter-update progress-reporter (point))) 
                    table)) 
          (error 
           (message "error happened near %d" (point)) 
           (error (error-message-string err))))) 
      table))