Avoid the chardet.universaldetector deprecation warning

chardet 7 moved UniversalDetector to chardet.detector and turned the
old chardet.universaldetector import path into a stub that raises a
DeprecationWarning on import. Projects that run their test suites with
warnings turned into errors (like Flexget, per the linked issue) hit a
hard failure the moment html5lib falls back to chardet for encoding
detection.

Try the new import path first and fall back to the old one, so this
keeps working across both old and new chardet releases without
triggering the warning on the new ones. Fixes #601.

Author: afonsojanu

AUTHORS.rst

@@ -63,4 +63,5 @@ Patches and suggestions
 - Ville Skyttä
 - Hugo van Kemenade
 - Mark Vasilkov
+- Afonso Januário
 

CHANGES.rst

@@ -20,6 +20,9 @@ Bug fixes:
 
 * The sanitizer now permits ``<summary>`` tags. It used to allow ``<details>``
   already. (#423)
+* Encoding detection no longer triggers a ``DeprecationWarning`` under
+  chardet 7+, which moved ``UniversalDetector`` out of
+  ``chardet.universaldetector``. (#601)
 
 1.1
 ~~~

html5lib/_inputstream.py

@@ -483,10 +483,16 @@ def determineEncoding(self, chardet=True):
         # Guess with chardet, if available
         if chardet:
             try:
-                from chardet.universaldetector import UniversalDetector
+                # chardet 7+ moved UniversalDetector here; the older
+                # chardet.universaldetector path still works but now raises
+                # a DeprecationWarning.
+                from chardet.detector import UniversalDetector
             except ImportError:
-                pass
-            else:
+                try:
+                    from chardet.universaldetector import UniversalDetector
+                except ImportError:
+                    UniversalDetector = None
+            if UniversalDetector is not None:
                 buffers = []
                 detector = UniversalDetector()
                 while not detector.done: