Your problem is no different from trying to figure out where paragraphs and columns are in PDF files; PDF does not usually place a table of contents page as such. Therefore, even with a PDF library (e.g. iTextSharp marked with mkl) this will not be a trivial task.
With such a library, you can see the pages in the PDF file and the text on the pages. However, if this is a book, for example, the table of contents page may be the first, second, third or x-page page in the PDF file due to the fact that other other pages appear in front of it (cover, second cover, copyright, you name it. ..).
Thus, an algorithm for detecting whether there is a table of contents would have to find it somewhere on the first x pages of the PDF file. Since there are no standard tags that highlight text in the table of contents, this should be done by analyzing the text format on this page.
There are two things that can help (if available):
1) In many PDF files, the elements in the table are content, as you say, you can click. Thus, you can look in the PDF file and try to find the first page containing many hyperlinks.
2) In many PDF files, the table of contents is mirrored in bookmarks. Thus, you can also study the structure of bookmarks and see if you can use this to find out how many chapters are in the book.
Keep in mind that both of these functions are optional and are not standardized if present.