Android Paging 3 library PagingSource invalidation, causes the list to jump due to wrong refresh key (not using room)

Since I'm currently working on a Project with custom database (not Room), I'm testing whether we could use the Paging 3 library in the Project. However, I run into the issue, that if you make changes to the data and therefore invalidate the paging source, the recreation of the list is buggy and jumps to a different location. This is happening because the Refresh Key calculation seems to be wrong, which is most likely caused by the fact that the initial load loads three pages worth of data, but puts it into one page.

The default Paging Source looks like this:

    override fun getRefreshKey(state: PagingState<Int, CustomData>): Int? {
        // Try to find the page key of the closest page to anchorPosition, from
        // either the prevKey or the nextKey, but you need to handle nullability
        // here:
        //  * prevKey == null -> anchorPage is the first page.
        //  * nextKey == null -> anchorPage is the last page.
        //  * both prevKey and nextKey null -> anchorPage is the initial page, so
        //    just return null.
        return state.anchorPosition?.let { anchorPosition ->
            val anchorPage = state.closestPageToPosition(anchorPosition)
            anchorPage?.prevKey?.plus(1) ?: anchorPage?.nextKey?.minus(1)
        }
    }

override suspend fun load(params: LoadParams<Int>): LoadResult<Int, CustomData> {
    var pagePosition = params.key ?: STARTING_PAGE_INDEX
    var loadSize = params.loadSize
    return try {
        val dataResult = dataRepository.getPagedData(
            pagePosition = pagePosition,
            loadSize = loadSize,
            pageSize = pageSize
        )
        val nextKey = if (dataResult.isEmpty() || dataResult.size < pageSize) {
            null
        } else {
            pagePosition + (loadSize / pageSize)
        }
        Log.i(
            "RoomFreePagingSource",
            "page $pagePosition with size $loadSize publish ${dataResult.size} routes"
        )
        return LoadResult.Page(
            data = dataResult,
            prevKey = if (pagePosition == STARTING_PAGE_INDEX) null else pagePosition - 1,
            nextKey = nextKey
        )

    } catch (exception: Exception) {
        LoadResult.Error(exception)
    }
}

The dataRepository.getPagedData() function simply accesses a in memory list and returns a subsection of the list, simulating the paged data. For completeness here is the implementation of this function:

fun getPagedData(pageSize:Int, loadSize:Int, pagePosition: Int): List<CustomData>{
    val startIndex = pagePosition * pageSize
    val endIndexExl =startIndex + loadSize
    return data.safeSubList(startIndex,endIndexExl).map { it.copy() }
}

private fun <T> List<T>.safeSubList(fromIndex: Int, toIndex: Int) : List<T>{
    // only returns list with valid range, to not throw exception
    if(fromIndex>= this.size)
        return emptyList()
    val endIndex = if(toIndex> this.size) this.size else toIndex
    return subList(fromIndex,endIndex)
}

The main Problem I currently face is, that the getRefreshKey function doesn't return a correct refresh page key, which results in the wrong page being refreshed and the list jumping to the loaded page.

• For example if you have 10 items per page.
• The first page 0 contains 30 items
• The next page would be 3 and contains 10 items.
• If you don't scroll (only see the first 7 items) and invalidate then the anchorPosition will be 7 and the refresh key will be 2. (anchorPage.prevKey = null => anchorPage.nextKey = 3 => 3-1 is 2) However, at this point we would like to load page 0 not page 2.

Knowing the cause of the issue I tried to adapt the default implementation to resolve it and came up with a lot of different versions. The one below currently works best, but still causes the jumping from time to time. And also sometimes a part of the list flickers, which is probably caused when not enough items are fetched to fill the view port.

override fun getRefreshKey(state: PagingState<Int, CustomData>): Int? {
    return state.anchorPosition?.let { anchorPosition ->
        val closestPage = state.closestPageToPosition(anchorPosition)?.prevKey
            ?: STARTING_PAGE_INDEX
        val refKey = if(anchorPosition>(closestPage)*pageSize + pageSize)
            closestPage+1
        else
            closestPage

        Log.i(
            "RoomFreePagingSource",
            "getRefreshKey $refKey from anchorPosition $anchorPosition closestPage $closestPage"
        )
        refKey
    }
}

override suspend fun load(params: LoadParams<Int>): LoadResult<Int, CustomData> {
    var pagePosition = params.key ?: STARTING_PAGE_INDEX
    var loadSize = pageSize
    return try {
        when (params) {
            is LoadParams.Refresh -> {
                if (pagePosition > STARTING_PAGE_INDEX) {
                    loadSize *= 3
                } else if (pagePosition == STARTING_PAGE_INDEX) {
                    loadSize *= 2
                }
            }
            else -> {}
        }
        val dataResult = dataRepository.getPagedData(
            pagePosition = pagePosition,
            loadSize = loadSize,
            pageSize = pageSize
        )
        val nextKey = if (dataResult.isEmpty() || dataResult.size < pageSize) {
            null
        } else {
            pagePosition + (loadSize / pageSize)
        }
        Log.i(
            "RouteRepository",
            "page $pagePosition with size $loadSize publish ${dataResult.size} routes"
        )
        return LoadResult.Page(
            data = dataResult,
            prevKey = if (pagePosition == STARTING_PAGE_INDEX) null else pagePosition - 1,
            nextKey = nextKey
        )

    } catch (exception: Exception) {
        LoadResult.Error(exception)
    }
}

In theory I found that the best solution would be to just calculate the refresh PageKey like this anchorPosition/pageSize then load this page, the one before and after it. Hower, calculating the refresh key like this does not work since the anchorPosition isn't the actual position of the item in the list. After some invalidations, the anchor could be 5 even if you are currently looking at item 140 in the list.

So to sum it up:

How can I calculate the correct refresh page after invalidation when I don't use Room, but another data source, like in this sample case a in memory list which I access through getPagedData?

from Android Paging 3 library PagingSource invalidation, causes the list to jump due to wrong refresh key (not using room)