By nature, the sitemap requires a deep but thin capture of most of the Topic tree. With the proposal for lazy-loading the CachedTopicRepository (see #111) that becomes much more expensive. Instead, introduce a barebones ISitemapTopicRepository with just a Load() method on it, plus a lightweight SqlSitemapTopicRepository that generates a lightweight, single-use Topic tree for use with the SitemapController.
Implementation
- The concrete
SqlSitemapTopicRepository can live alongside the existing SqlTopicRepository in the OnTopic.Data.Sql project
- This should call a new stored procedure, such as
GetSitemap, as an alternative to GetTopics
- This will still return all topics from
Topics for the first data set
- This will just return the
IsPrivateBranch', 'NoIndex', 'IsDisabled', 'Url', and 'LastModified attributes from AttributeIndex
- This will not return extended attributes, relationships, or versions
- This can still be assembled using the existing
LoadTopicGraph() and GetTopics() extension methods on SqlDataReaderExtensions
- Since this is relying on the
SqlDataReaderExtensions and returning a Topic tree, the existing SitemapController will mostly work as is
- It will need a dependency on the
ISitemapTopicRepository introduced
- The
Extended action support was meant for the legacy Bing Search API, and can now be removed entirely
- No current major sitemap.org consumers index arbitrary key/value pairs, as Bing Search API did
- We may want to support schema.org extensions later, but that is a different project, and requires mapping attributes to schema.org structures
- This removes the dependency on
ITopicRepository, and allows quite a bit of plumbing to be removed from the SitemapController
- This includes the
getAttributes(), getRelationships(), and getReferences() page map helpers
- This includes the
includeMetadata parameter as well as its conditional blocks
- We will want to provide output caching so we don't need to keep querying the data
- Based on e.g., GoldSim's Azure Application Insights data, their sitemap gets queried every ≈15 minutes
- The sitemap doesn't need to be realtime, and can be aggressively cached for performance reasons
- We can use .NET 7+'s
AddOutputCache(), UseOutputCache(), and .CacheOutput for this instead of Response Caching
Limitations
For this version, we will not be supporting:
- Pre-filtering data in SQL by
IsPrivateBranch as well as NoIndex on Container content type
- This is a possible future enhancement since in e.g., GoldSim only 3 out of 9 of their root nodes fit this criteria
- That said, with a light data load and output caching, this shouldn't be a significant performance boost
- Implementing
IReadOnlyTopicRepository, which is a similar pattern where we'll have a light-weight read-only interface that doesn't permit writing back to the underlying data source
- This would similarly just support
Load(), but will likely support additional overloads
- This will likely just be added to existing implementations to restrict operations performed on them, as opposed to requiring new implementations as is the case here
By nature, the sitemap requires a deep but thin capture of most of the Topic tree. With the proposal for lazy-loading the
CachedTopicRepository(see #111) that becomes much more expensive. Instead, introduce a barebonesISitemapTopicRepositorywith just aLoad()method on it, plus a lightweightSqlSitemapTopicRepositorythat generates a lightweight, single-use Topic tree for use with theSitemapController.Implementation
SqlSitemapTopicRepositorycan live alongside the existingSqlTopicRepositoryin theOnTopic.Data.SqlprojectGetSitemap, as an alternative toGetTopicsTopicsfor the first data setIsPrivateBranch', 'NoIndex', 'IsDisabled', 'Url', and 'LastModifiedattributes fromAttributeIndexLoadTopicGraph()andGetTopics()extension methods onSqlDataReaderExtensionsSqlDataReaderExtensionsand returning a Topic tree, the existingSitemapControllerwill mostly work as isISitemapTopicRepositoryintroducedExtendedaction support was meant for the legacy Bing Search API, and can now be removed entirelyITopicRepository, and allows quite a bit of plumbing to be removed from theSitemapControllergetAttributes(),getRelationships(), andgetReferences()page map helpersincludeMetadataparameter as well as its conditional blocksAddOutputCache(),UseOutputCache(), and.CacheOutputfor this instead of Response CachingLimitations
For this version, we will not be supporting:
IsPrivateBranchas well asNoIndexonContainercontent typeIReadOnlyTopicRepository, which is a similar pattern where we'll have a light-weight read-only interface that doesn't permit writing back to the underlying data sourceLoad(), but will likely support additional overloads