Monthly Archives: August 2013

Solution for SqlDataReader.ReadColumnHeader NullReferenceException

This post related to the previous one, but I decided to write a separate article because it seems to be a common problem.

Sometimes when you use SqlDataReader, you would get an exception:

NullReferenceException {“Object reference not set to an instance of an object.”}
at System.Data.SqlClient.SqlDataReader.ReadColumnHeader(Int32 i)
at System.Data.SqlClient.SqlDataReader.ReadColumn(Int32 i, Boolean setTimeout)
at System.Data.SqlClient.SqlDataReader.GetInt32(Int32 i)

And the maddening thing – it doesn’t happen often, just every once in a while. And it happens at different times too, sometimes reader would read 100 records, sometimes 200 etc.

One possible case – SqlDataReader is losing its connection. And one possible reason for that – connection goes out of scope.

Consider following scenario – you have a function that returns SqlDataReader:

Function GetTheReader() as SqlDataReader
   Dim oConn As New SqlConnection("Connection String") : oConn.Open()
   Dim oComm As New SqlCommand("Stored Procedure", oConn)

   Dim oReader As SqlDataReader = oComm.ExecuteReader(CommandBehavior.CloseConnection)

   Return oReader
End Function

And you use it like this:

Dim oReader as SqlDataReader = GetTheReader()
'Begin use reader - loop, read data etc.

The problem with this approach that connection used to create the reader is stored in a private variable inside of `GetTheReader` function and when the function exits – the variable goes out of scope. Eventually, sooner or later Garbage Collector will collect it and close and dispose of connection – and at this time your SqlDataReader will fail.

The solution? Either use SqlDataReader at the same scope level you created it, or, if you do need to use function – pass connection object into it as one of the parameters, so it would remain valid after function exits.

When SqlDataReader is missing rows

I have a very basic scenario:

  1. Execute TSQL Stored procedure
  2. Return a DataReader
  3. Read data from the Reader

This is ADO.NET 101. There is one problem: DataReader loses rows. This problem has haunted me forever, extensive research and numerous suggestion didn’t help, even though the code is extremely basic:

Get the reader:

m_dbSel.CommandType = CommandType.StoredProcedure
m_dbSel.CommandText = "SP_Name"
oResult = m_dbSel.ExecuteReader()

Pass the reader to class constructor to fill Generic List (of Integer):

Public Sub New(i_oDataReader As Data.SqlClient.SqlDataReader)

    m_aFullIDList = New Generic.List(Of Integer)

    While i_oDataReader.Read
        m_aFullIDList.Add(i_oDataReader.GetInt32(0))
    End While

    m_iTotalNumberOfRecords = m_aFullIDList.Count

End Sub

This problem occurs when number of rows returned by the reader is relatively large (over 600,000 records). If this happens – number of rows added to the list from the reader is inconsistent, but always less than real one. Most often “magic” number of 524289 rows is returned.

Well, this is no longer a mystery, thanks to the great people from Stack Overflow @RBarryYoung, @granadaCoder and especially @MartinSmith who was the first to point me in the right direction – and here it is.

Even though the problem is with SqlDataReader – it is happening because it is used in conjunction with Generic List. List, as you may know has a flexible Capacity for number of elements it can store. When count of elements exceeds capacity – capacity increases and always to a power of 2. E.g.

When the count exceeds 4 elements – capacity is set to 8 (2^3)
When the count exceeds 8 elements – capacity is set to 16 (2^4)
When the count exceeds 16 elements – capacity is set to 32 (2^5)

etc..

This is what makes Generic List such a powerful tool, used by many large scale .NET projects, e.g. bingogodz.com. And ordinary this is not a problem. Unfortunately this is not the case when it is used together with SqlDataReader. When count of items in the List exceeds 524,288 (2^19) and its capacity is set to 1,048,576 (2^20) – SqlDataReader’s Read method suddenly returns False even though not all records have been read. No exception is thrown, it simple stops.

The only possible workaround I’ve found so far (I am still looking for better ones) is to pre-set List capacity in advance. Since, when using DataReader, you do not know number of rows, you’re left either with hardcoding the number or running another DB query to retrieve number of rows via something like COUNT(*). Like I said, I don’t like this workaround, please let me know if you find a better one.


UPDATE: Finally figured it out: http://stackoverflow.com/a/18520609/961695

WebHierarchicalDataGrid: get_scrollTop() returns incorrect value

Infragistics WebHierarchicalDataGrid has a neat client-side built-in function get_scrollTop() – it is used if at any point you need to retrieve current vertical scroll position of the grid (e.g. to use it in your own calculations to display something at a specific position on the grid – tooltip, help, additional info etc.)

Unfortunately the function has a bug: its value only set if user actually manually scrolls the grid: using mouse and scrollbar on the right, keyboard etc. If no scrolling user-interaction is involved and scroll position changes due to other means (e.g. displayed data size changes) – the function retains original value, throwing all your calculation out of whack. Continue reading →

ADO.NET DataTable: Change Column DataType after table is populated with data

Sometimes there is a need to change DataType of ADO.NET DataTable column. If your table is populated as a result of some database operation – you don’t know in advance what type the columns will be. And by design you cannot change the type of the column after the table is populated. Conudrum. Catch 22. Tough luck.

But wait, there’s light at the end of the tunnel. You cannot change the type of the existing column, but you can create a new one. Continue reading →