How to send larger than 4k queries from SQL buffer to sql-mysql buffer in Emacs?

Question

I've frequently run into an annoyance in Emacs's sql-mysql mode, and I'm wondering if anyone has a solution or better workaround. Anytime I try to send a query from an sql-mode buffer to an active SQL process buffer, that query cannot be larger than 4k. If it's larger than 4k, it appears that some sort of break - perhaps a newline - is inserted, and this causes the mysql interpreter to throw an error on the following line.

sql-mysql is implemented by sql.el, and uses the function sql-send-region to send query regions (or whole buffers) to the selected SQL process buffer. sql-send-region calls comint-send-region, which in turn calls process-send-region. process-send-region is a C function that calls send_process, both in src/process.c in the Emacs source.

It looks like this may just be a limitation produced by the 4k buffer on an IPC pipe. Since it appears that kernel hacking is necessary to change this size, that's not a great answer.

What I guess I'm puzzled by is why the SQL sent through the pipe is not properly reassembled by the mysql client if it's larger than 4k. Any ideas?

Emacs version: GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.10) of 2012-03-25 on allspice, modified by Debian

mysql -V: mysql Ver 14.14 Distrib 5.5.24, for debian-linux-gnu (x86_64) using readline 6.2

Sql Mysql Options: -A -C -n (NB I've tried both with and without -n (unbuffered) and neither fixed this issue)

BTW, the best workaround I've so far found is to source the query directly from the saved file, i.e. "\. sql_long_query.sql". However, this is rather annoying when trying to debug portions of a query or multi-query files. That's why I'd like to be able to use the sql-send-region method even for regions larger than 4k chars. — Ryan M
Also note that simply copying and pasting a query of longer than 4k into the sql-mysql buffer encounters the same issue. More evidence that the problem is at or below the send_process C func level. — Ryan M
@ramrunner, that would require emacs to be run from a terminal, which isn't always (practically never, imo) desirable. — Squidly
@MrBones: i don't get it. couldn't you change your "button" in the gui to execute "rlwrap emacs"? that's trivial. — ramrunner

user4815162342 user4815162342 · Accepted Answer · 2012-09-10T20:52:19

I suspect the culprit to be Emacs's process communication code, used by comint, allocating PTYs to talk to processes. While these are useful for interactive work because they allow job control, they are also limited in how much data they can transfer in one chunk without an intervening newline. Emacs can also be told to use pipes, which do not have this limitation.

To test this, start a fresh Emacs, evaluate M-: (setq process-connection-type nil), and start sql-mysql. If the problem goes away, this is your culprit. In that case, you will want to use something like:

(add-hook 'sql-mysql-mode-hook
          (lambda ()
            (set (make-local-variable 'process-connection-type) nil)))

to make sure that process-connection-type gets reset only in MySQL interaction buffers.

EDIT

According to http://tinyurl.com/car439o/, Emacs no longer bothers to interrupt long PTY output with newline+EOF pairs. Although the commit is from 2010-04-13, it only appeared in Emacs 24, released in 2012. This would explain why the problem is apparently not reproducible in 24.2.1. If you are using Emacs prior to 24, try upgrading.

How to send larger than 4k queries from SQL buffer to sql-mysql buffer in Emacs?

1 Answers