Monday, 30 December 2019

Pandas group by result to columns

I have dataframe like this:

x = pd.DataFrame({
    'audio': ['audio1', 'audio1', 'audio2', 'audio2', 'audio3', 'audio3'],
    'text': ['text1', 'text2', 'text3', 'text4', 'text5', 'text6'],
    'login': ['operator1', 'operator2', 'operator3', 'operator4', 'operator5', 'operator6'] 
})

i'm trying to aggregate it like this:

x1 = x.groupby('audio')['text'].agg(
    [
    ('text1', lambda x : x.iat[0]),
    ('text2', lambda x : x.iat[1]),
    ('leven', lambda x: Levenshtein.distance(x.iat[0], x.iat[1])) #some function works with grouped text
    ]
).reset_index()

and it works but i also need to add grouped logins to row, to make row like this:

audio, text1, text2, leven, login1, login2

I tried something like lambda x : x.ait[0, 1] but it doesnt work



from Pandas group by result to columns

WAP_PUSH_DELIVER Intent never sent by Android system

I spent 5 hours attempting to make a custom MMS Broadcast receiver work. I googled and searched here at Stackoverflow a gazillion of seemingly duplicate threads but non of the workarounds or proposed solutions.

A few random debugging notes:

  • The app is set as default SMS app and all permissions were granted.
  • Verified that RCS is disabled through Google's Messenger.
  • The DUT is a Pixel 3 XL with latest software updates.
  • Tested many 3P SMS/MMS apps, almost all aren't able to receive MMS either, however, Pulse does. Pulse's developer provides an Open Source MMS library. I tried leveraging this library without success. The core problem seems to be that WAP_PUSH_DELIVER Intents are not sent to my application.
  • Decoded the Pulse app to have a look at its AndroidManifest.xml and try a few different changes in mine. No success.
  • Test MMS were sent with Google Voice account via legacy Hangouts and from other Android phones via various carriers.
  • SMS can be received without problems both SMS_RECEIVED and SMS_DELIVER (if the app is the default SMS one).
  • WAP_PUSH_RECEIVED events are received if the app is not the default SMS one
  • WAP_PUSH_DELIVER events are not received if the app is the default SMS one
  • Installed Intent Intercepter and ran $ adb logcat | fgrep -i intent to verify if any permission problems are logged. None are.

MmsReceiver:

public class MmsReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        System.out.println("MMS Received: " + intent.getAction());
    }
}

Manifest:

<uses-permission android:name="android.permission.SEND_SMS" />
<uses-permission android:name="android.permission.READ_SMS" />
<uses-permission android:name="android.permission.WRITE_SMS" />
<uses-permission android:name="android.permission.RECEIVE_SMS" />
<uses-permission android:name="android.permission.RECEIVE_MMS" />
<uses-permission android:name="android.permission.BROADCAST_WAP_PUSH" />
<uses-permission android:name="android.permission.BROADCAST_SMS" />
<uses-permission android:name="android.permission.RECEIVE_WAP_PUSH" />
<uses-permission android:name="android.permission.WRITE_APN_SETTINGS" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
<uses-permission android:name="android.provider.Telephony.SMS_RECEIVED" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_SETTINGS"/>


    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.SEND" />
                <action android:name="android.intent.action.SENDTO" />

                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />

                <data android:scheme="sms" />
                <data android:scheme="smsto" />
                <data android:scheme="mms" />
                <data android:scheme="mmsto" />
            </intent-filter>
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>

        <receiver
            android:name=".SmsReceiver"
            android:permission="android.permission.BROADCAST_SMS">
            <intent-filter>
                <action android:name="android.provider.Telephony.SMS_DELIVER" />
            </intent-filter>
        </receiver>

        <receiver
            android:name=".MmsReceiver"
            android:permission="android.permission.BROADCAST_WAP_PUSH">
            <intent-filter>
                <action android:name="android.provider.Telephony.WAP_PUSH_DELIVER" />
                <data android:mimeType="application/vnd.wap.mms-message" />
            </intent-filter>
        </receiver>

        <service
            android:name=".HeadlessSmsSendService"
            android:exported="true"
            android:permission="android.permission.SEND_RESPOND_VIA_MESSAGE" >
            <intent-filter>
                <action android:name="android.intent.action.RESPOND_VIA_MESSAGE" />
                <category android:name="android.intent.category.DEFAULT" />

                <data android:scheme="sms" />
                <data android:scheme="smsto" />
                <data android:scheme="mms" />
                <data android:scheme="mmsto" />
            </intent-filter>
        </service>
    </application>

</manifest>

Needless to mention, the message is not logged in Logcat.



from WAP_PUSH_DELIVER Intent never sent by Android system

python - asn1 parsed text to json

With text given in this link, need to extract data as follows

  1. Each record starts with YYYY Mmm dd hh:mm:ss.ms, for example 2019 Aug 31 09:17:36.550
  2. Each record has a header starting from line #1 above and ending with a blank line
  3. The record data is contained in lines below Interpreted PDU:
  4. The records of interest are the ones with record header first line having 0xB821 NR5G RRC OTA Packet -- RRC_RECONFIG

Is it possible to extract selected record headers and text below #3 above as an array of nested json in the format as below - snipped for brevity, really need to have the entire text data as JSON.

data = [{"time": "2019 Aug 31  09:17:36.550", "PDU Number": "RRC_RECONFIG Message", "Physical Cell ID": 0, "rrc-TransactionIdentifier": 1, "criticalExtensions rrcReconfiguration": {"secondaryCellGroup": {"cellGroupId": 1, "rlc-BearerToAddModList": [{"logicalChannelIdentity": 1, "servedRadioBearer drb-Identity": 2, "rlc-Config am": {"ul-AM-RLC": {"sn-FieldLength": "size18", "t-PollRetransmit": "ms40", "pollPDU": "p32", "pollByte": "kB25", "maxRetxThreshold": "t32"}, "dl-AM-RLC": {"sn-FieldLength": "size18", "t-Reassembly": "ms40", "t-StatusProhibit": "ms20"}}}]}}  }, next records data here]

Note that the input text is parsed output of ASN1 data specifications in 3GPP 38.331 section 6.3.2. I'm not sure normal python text parsing is the right way to handle this or should one use something like asn1tools library ? If so an example usage on this data would be helpful.



from python - asn1 parsed text to json