partitioning big tables - indexes

Question

i ve got assigned a task to improve data management proces(data archivization) on couples of tables tables are like 200gb

i am reading now about table partitioning and best practices and as far as i know now the process goes like

create filegroups and files
create partitioning function
partitioning scheme - (map intervals to appopriate filegroups)
recreate clustered index - this is the moment when table is physically moved to another files
profit :)

but cant find one information what is going on with existing non clustered indexes at this point ? from here : http://technet.microsoft.com/en-us/library/ms187526(v=sql.105).aspx i have found

Although partitioned indexes can be implemented independently from their base tables, it generally makes sense to design a partitioned table and then create an index on the table. When you do this, SQL Server automatically partitions the index by using the same partition scheme and partitioning column as the table. As a result, the index is partitioned in essentially the same manner as the table. This makes the index aligned with the table.

and another one

When partitioning a unique nonclustered index, the index key must contain the partitioning column. When partitioning a nonunique, nonclustered index, SQL Server adds the partitioning column by default as a nonkey (included) column of the index to make sure the index is aligned with the base table. SQL Server does not add the partitioning column to the index if it is already present in the index.

but none of this reference my problem do i have to create partitioning function explicitly for existing non clustered indexes which does/doesnt have partitioning column in their definition ?

lets say we have table like

table A - col1 col2 col3

with clustered index on col1 and non clustered on col 3 on PRIMARY partition

what is going to happen with non clustered index on col3 after partitioning , is it going to be aligned with table or still resides on PRIMARY partition

Remus Rusanu Remus Rusanu · Accepted Answer · 2012-08-07T08:44:17

You should align your indexes. There are two main forces pulling you into this direction:

fast partition switch operations require aligned indexes. When dealing with large data sets deleting obsolete data (data that has passed the required retention period) is just as important as adding new data. Partition switch operations are by far the most efficient way of deleting old data. See How to Implement an Automatic Sliding Window in a Partitioned Table
query processor loves aligned indexes and hates the unaligned ones. For example see Memory Limitations and Partitioned Indexes:

For both aligned and nonaligned indexes, the memory requirement can be greater if SQL Server is applying degrees of parallelism to the build operation on a multiprocessor computer. This is because the greater the degrees of parallelism, the greater the memory requirement. For example, if SQL Server sets degrees of parallelism to 4, a nonaligned partitioned index with 100 partitions requires sufficient memory for four processors to sort 4,000 pages at the same time, or 16,000 pages. If the partitioned index is aligned, the memory requirement is reduced to four processors sorting 40 pages, or 160 (4 * 40) pages.

In your case this implies adding the partitioning column explicitly to each non-clustered index and declare each non-clustered index on the same partition scheme as the base table (the clustered index). Do no attempt to create a different partition function/scheme for the non-clustered index. Adding the partitioning column to each index has deep repercussions in your data model, eg. you will no longer be able to declare a primary key constraint that does not contain the partitioning column (and this ripples into all the foreign key definitions referencing the partitioned table!) but this is a price you already bought into when you accepted partitioning as a solution, see How To Decide if You Should Use Table Partitioning.

partitioning big tables - indexes

2 Answers