Webflow's CMS Lists do not have a native "grouping" feature, however with a bit of ingenuity and the use of conditional visibility, it's possible to create neatly grouped lists.
If you like, you can further add in-page navigation to make it easily accessible.
In this tutorial, we'll build a standard A-Z Glossary with letter grouping and a navigation bar.
Webflow's site has a page with this type of item grouping here.
This demonstration is centered on an alphabetic grouping, and has an easily identifiable distinction between group-header-items and content-type items, in the presence or absence of description content.
Despite that, this grouping technique should be adaptable to other CMS list grouping use-cases as well.
There are only three key mechanics at work;
Pagination will mostly work, however if you are on page 2, the heading will not be repeated on that page. You'd see the content items, without a group header, until the next group begins.