Mysql Connection throwing a Exception when get record is more than 1 million

Question

I am facing the problem with MySqlClient connection. When I am getting more than 1 million records, it is throwing an exception. The same code is working fine when the record count is in thousands.

Stack trace:

at MySql.Data.MySqlClient.MySqlField..ctor(MySqlConnection connection) at MySql.Data.MySqlClient.NativeDriver.GetFieldMetaData41() at MySql.Data.MySqlClient.NativeDriver.GetFieldMetaData() at MySql.Data.MySqlClient.NativeDriver.ReadColumnMetadata(Int32 count) at MySql.Data.MySqlClient.ResultSet.LoadMetaData() at MySql.Data.MySqlClient.ResultSet.NextResult() at MySql.Data.MySqlClient.MySqlDataReader.NextResult() at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior) at MySql.Data.MySqlClient.MySqlCommand.ExecuteDbDataReader(CommandBehavior behavior) at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior) at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior) at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior) at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet) at _Default.Button1_Click(Object sender, EventArgs e)

Thanks Atul

Answer 1

You are probably just running out of memory.

Answer 2

I agree with Rick: you're running out of memory on the machine.. Probably your web server.

I'd highly suggest you seek an alternative method for pulling 1 million + records out of your database. Data Adapters are already resource intensive and there are much better ways.

The first thing is to decide whether you even need that many. You indicated "asp.net" so this is a web app. I can't think of any reason for a web server to pull that many records unless you are doing some type of mass analysis at which point you should investigate what features are available in your database server that are better suited to handle it.