Android Navigation Architecture Component - Nav Drawer Icons

I'm currently using the Android Architecture Component's Navigation, but I'm running into an issue with my Navigation Drawer. It shows the hamburger menu when at my starting destination, but other Fragments are showing the up arrow. I believe I've setup my navigation_graph incorrectly.

navigation.graph.xml

<?xml version="1.0" encoding="utf-8"?>
<navigation xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    app:startDestination="@id/nav_home">

    <!-- Start at HomeFragment -->
    <fragment
        android:id="@+id/nav_home"
        android:name=".HomeFragment"
        android:label="@string/home">

        <!-- Navigate to the Search -->
        <action
            android:id="@+id/action_nav_home_to_nav_search"
            app:destination="@id/nav_search" />
    </fragment>


    <fragment
        android:id="@+id/nav_settings"
        android:name=".SettingsFragment"
        android:label="@string/settings">

        <!-- Navigate to the Search -->
        <action
            android:id="@+id/action_nav_settings_to_nav_search"
            app:destination="@id/nav_search" />
    </fragment>



    <fragment
        android:id="@+id/nav_search"
        android:name=".SearchFragment"
        android:label="@string/search" />

</navigation>

I feel like HomeFragment and SettingsFragment should be related somehow but I'm not sure how to define that.

And then within MainActivity, I just set it up like this.

navController = findNavController(R.id.mainNavigationFragment)
setupActionBarWithNavController(this, navController, drawer_layout)

Thanks.

from Android Navigation Architecture Component - Nav Drawer Icons