Issue with combining MO segments with different encodings

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue with combining MO segments with different encodings

Nic Pottier-2
Hi all,

Use Kannel quite a bit, thanks for all the work. I've run into an
interesting issue which I'm not sure whether to blame on Kannel or the
SMSC. When a multi-part message comes through where some parts are
GSM7 and some parts are UTF16, Kannel combines the parts naively as
bytes and therefore leads to a corrupt message.

Example below with 4 parts. Note that part 1 has a coding of 8, while
the others are plain GSM7 with coding of 0. But when the final part is
received Kannel simply concatenates the bytes from all the parts,
leading to a mix of single and multi-byte message, which is obviously
corrupt. (immediately below)

Is this valid SMPP traffic? (to use separate encodings for each of the
parts of a segmented message) I do notice that the GSM7 parts (2-4)
while definitely encoded as GSM7 are only 73 characters long which
seems suspicious. At the same time, it seems Kannel should be
converting these all to unicode internally and then appending them
instead of just appending the bytes. So could see this being an
argument that Kannel is wrong here.

This is on version 1.4.4

Octet string at 0x7f191880ebe0:
  len:  283
  size: 1024
  immutable: 0
  data: 00 45 00 75 00 20 00 74 00 69 00 76 00 20 00 75   .E.u. .t.i.v. .u
  data: 00 6d 00 61 00 20 00 72 00 65 00 6c 00 61 00 e7   .m.a. .r.e.l.a..
  data: 00 61 00 6f 00 20 00 64 00 65 00 73 00 70 00 72   .a.o. .d.e.s.p.r
  data: 00 6f 00 74 00 65 00 67 00 69 00 64 00 61 00 20   .o.t.e.g.i.d.a.
  data: 00 6e 00 6f 00 20 00 6d 00 65 00 73 00 20 00 70   .n.o. .m.e.s. .p
  data: 00 61 00 73 00 73 00 61 00 64 00 6f 00 20 00 65   .a.s.s.a.d.o. .e
  data: 00 20 00 f1 00 20 00 76 00 69 00 20 00 6d 00 65   . ... .v.i. .m.e
  data: 00 73 00 74 00 72 00 75 00 61 00 e7 00 61 00 6f   .s.t.r.u.a...a.o
  data: 00 20 00 70 00 6f 72 71 20 65 72 61 20 61 6e 74   . .p.orq era ant
  data: 65 73 20 64 65 20 76 65 72 20 6e 61 71 75 65 6c   es de ver naquel
  data: 65 20 6d 65 73 20 6d 61 73 20 64 65 70 6f 69 73   e mes mas depois
  data: 20 6e 6f 20 69 6e 69 63 69 6f 20 64 65 73 74 65    no inicio deste
  data: 20 65 6d 20 63 75 72 73 6f 20 76 69 20 75 6d 61    em curso vi uma
  data: 73 20 67 6f 74 61 73 20 64 65 20 73 61 6e 67 75   s gotas de sangu
  data: 65 20 6d 61 73 20 6d 75 69 74 6f 20 63 61 72 72   e mas muito carr
  data: 65 67 61 64 61 20 64 75 72 61 6e 74 65 20 64 6f   egada durante do
  data: 69 73 20 64 69 61 73 2e 73 65 72 61 71 20 73 74   is dias.seraq st
  data: 6f 75 20 67 72 61 76 69 64 61 3f                  ou gravida?
Octet string dump ends.

------------------------------------------------------------------------
Part 1 delivered:

SMPP PDU 0x7f191880cb40 dump:
  type_name: deliver_sm
  command_id: 5 = 0x00000005
  command_status: 0 = 0x00000000
  sequence_number: 294621 = 0x00047edd
  service_type: NULL
  source_addr_ton: 1 = 0x00000001
  source_addr_npi: 1 = 0x00000001
  source_addr: "258844096933"
  dest_addr_ton: 0 = 0x00000000
  dest_addr_npi: 1 = 0x00000001
  destination_addr: "92222"
  esm_class: 64 = 0x00000040
  protocol_id: 0 = 0x00000000
  priority_flag: 0 = 0x00000000
  schedule_delivery_time: NULL
  validity_period: NULL
  registered_delivery: 0 = 0x00000000
  replace_if_present_flag: 0 = 0x00000000
  data_coding: 8 = 0x00000008
  sm_default_msg_id: 0 = 0x00000000
  sm_length: 140 = 0x0000008c
  short_message:
   Octet string at 0x7f191880cd70:
     len:  140
     size: 141
     immutable: 0
     data: 05 00 03 0d 04 01 00 45 00 75 00 20 00 74 00 69   .......E.u. .t.i
     data: 00 76 00 20 00 75 00 6d 00 61 00 20 00 72 00 65   .v. .u.m.a. .r.e
     data: 00 6c 00 61 00 e7 00 61 00 6f 00 20 00 64 00 65   .l.a...a.o. .d.e
     data: 00 73 00 70 00 72 00 6f 00 74 00 65 00 67 00 69   .s.p.r.o.t.e.g.i
     data: 00 64 00 61 00 20 00 6e 00 6f 00 20 00 6d 00 65   .d.a. .n.o. .m.e
     data: 00 73 00 20 00 70 00 61 00 73 00 73 00 61 00 64   .s. .p.a.s.s.a.d
     data: 00 6f 00 20 00 65 00 20 00 f1 00 20 00 76 00 69   .o. .e. ... .v.i
     data: 00 20 00 6d 00 65 00 73 00 74 00 72 00 75 00 61   . .m.e.s.t.r.u.a
     data: 00 e7 00 61 00 6f 00 20 00 70 00 6f               ...a.o. .p.o
   Octet string dump ends.
SMPP PDU dump ends.
SMPP[unicef_mz_vodacom]: UDH length read as 6
Got part 1 [ref 13, total parts 4] of message from +258844096933. Dump follows:

------------------------------------------------------------------------
Part 2 delivered:

SMPP PDU 0x7f1918809240 dump:
  type_name: deliver_sm
  command_id: 5 = 0x00000005
  command_status: 0 = 0x00000000
  sequence_number: 294618 = 0x00047eda
  service_type: NULL
  source_addr_ton: 1 = 0x00000001
  source_addr_npi: 1 = 0x00000001
  source_addr: "258844096933"
  dest_addr_ton: 0 = 0x00000000
  dest_addr_npi: 1 = 0x00000001
  destination_addr: "92222"
  esm_class: 64 = 0x00000040
  protocol_id: 0 = 0x00000000
  priority_flag: 0 = 0x00000000
  schedule_delivery_time: NULL
  validity_period: NULL
  registered_delivery: 0 = 0x00000000
  replace_if_present_flag: 0 = 0x00000000
  data_coding: 0 = 0x00000000
  sm_default_msg_id: 0 = 0x00000000
  sm_length: 73 = 0x00000049
  short_message:
   Octet string at 0x7f1918811130:
     len:  73
     size: 74
     immutable: 0
     data: 05 00 03 0d 04 02 72 71 20 65 72 61 20 61 6e 74   ......rq era ant
     data: 65 73 20 64 65 20 76 65 72 20 6e 61 71 75 65 6c   es de ver naquel
     data: 65 20 6d 65 73 20 6d 61 73 20 64 65 70 6f 69 73   e mes mas depois
     data: 20 6e 6f 20 69 6e 69 63 69 6f 20 64 65 73 74 65    no inicio deste
     data: 20 65 6d 20 63 75 72 73 6f                         em curso
   Octet string dump ends.
SMPP PDU dump ends.
SMPP[unicef_mz_vodacom]: UDH length read as 6
Got part 2 [ref 13, total parts 4] of message from +258844096933. Dump follows:

------------------------------------------------------------------------
Part 3 delivered:

SMPP PDU 0x7f1918809240 dump:
  type_name: deliver_sm
  command_id: 5 = 0x00000005
  command_status: 0 = 0x00000000
  sequence_number: 294612 = 0x00047ed4
  service_type: NULL
  source_addr_ton: 1 = 0x00000001
  source_addr_npi: 1 = 0x00000001
  source_addr: "258844096933"
  dest_addr_ton: 0 = 0x00000000
  dest_addr_npi: 1 = 0x00000001
  destination_addr: "92222"
  esm_class: 64 = 0x00000040
  protocol_id: 0 = 0x00000000
  priority_flag: 0 = 0x00000000
  schedule_delivery_time: NULL
  validity_period: NULL
  registered_delivery: 0 = 0x00000000
  replace_if_present_flag: 0 = 0x00000000
  data_coding: 0 = 0x00000000
  sm_default_msg_id: 0 = 0x00000000
  sm_length: 73 = 0x00000049
  short_message:
   Octet string at 0x7f191880be80:
     len:  73
     size: 74
     immutable: 0
     data: 05 00 03 0d 04 03 20 76 69 20 75 6d 61 73 20 67   ...... vi umas g
     data: 6f 74 61 73 20 64 65 20 73 61 6e 67 75 65 20 6d   otas de sangue m
     data: 61 73 20 6d 75 69 74 6f 20 63 61 72 72 65 67 61   as muito carrega
     data: 64 61 20 64 75 72 61 6e 74 65 20 64 6f 69 73 20   da durante dois
     data: 64 69 61 73 2e 73 65 72 61                        dias.sera
   Octet string dump ends.
SMPP PDU dump ends.
SMPP[unicef_mz_vodacom]: UDH length read as 6
Got part 3 [ref 13, total parts 4] of message from +258844096933. Dump follows:

------------------------------------------------------------------------
Part 4 delivered:

SMPP PDU 0x7f1918809240 dump:
  type_name: deliver_sm
  command_id: 5 = 0x00000005
  command_status: 0 = 0x00000000
  sequence_number: 294608 = 0x00047ed0
  service_type: NULL
  source_addr_ton: 1 = 0x00000001
  source_addr_npi: 1 = 0x00000001
  source_addr: "258844096933"
  dest_addr_ton: 0 = 0x00000000
  dest_addr_npi: 1 = 0x00000001
  destination_addr: "92222"
  esm_class: 64 = 0x00000040
  protocol_id: 0 = 0x00000000
  priority_flag: 0 = 0x00000000
  schedule_delivery_time: NULL
  validity_period: NULL
  registered_delivery: 0 = 0x00000000
  replace_if_present_flag: 0 = 0x00000000
  data_coding: 0 = 0x00000000
  sm_default_msg_id: 0 = 0x00000000
  sm_length: 21 = 0x00000015
  short_message:
   Octet string at 0x7f1918800d30:
     len:  21
     size: 22
     immutable: 0
     data: 05 00 03 0d 04 04 71 20 73 74 6f 75 20 67 72 61   ......q stou gra
     data: 76 69 64 61 3f                                    vida?
   Octet string dump ends.
SMPP PDU dump ends.
SMPP[unicef_mz_vodacom]: UDH length read as 6
Got part 4 [ref 13, total parts 4] of message from +258844096933. Dump follows:

Reply | Threaded
Open this post in threaded view
|

Re: Issue with combining MO segments with different encodings

amalysh
Hi,

as far as I know, tis is not allowed to ave multiple parts in different data coding.

Thanks,
Alex

> Am 22.02.2018 um 00:20 schrieb Nic Pottier <[hidden email]>:
>
> Hi all,
>
> Use Kannel quite a bit, thanks for all the work. I've run into an
> interesting issue which I'm not sure whether to blame on Kannel or the
> SMSC. When a multi-part message comes through where some parts are
> GSM7 and some parts are UTF16, Kannel combines the parts naively as
> bytes and therefore leads to a corrupt message.
>
> Example below with 4 parts. Note that part 1 has a coding of 8, while
> the others are plain GSM7 with coding of 0. But when the final part is
> received Kannel simply concatenates the bytes from all the parts,
> leading to a mix of single and multi-byte message, which is obviously
> corrupt. (immediately below)
>
> Is this valid SMPP traffic? (to use separate encodings for each of the
> parts of a segmented message) I do notice that the GSM7 parts (2-4)
> while definitely encoded as GSM7 are only 73 characters long which
> seems suspicious. At the same time, it seems Kannel should be
> converting these all to unicode internally and then appending them
> instead of just appending the bytes. So could see this being an
> argument that Kannel is wrong here.
>
> This is on version 1.4.4
>
> Octet string at 0x7f191880ebe0:
>  len:  283
>  size: 1024
>  immutable: 0
>  data: 00 45 00 75 00 20 00 74 00 69 00 76 00 20 00 75   .E.u. .t.i.v. .u
>  data: 00 6d 00 61 00 20 00 72 00 65 00 6c 00 61 00 e7   .m.a. .r.e.l.a..
>  data: 00 61 00 6f 00 20 00 64 00 65 00 73 00 70 00 72   .a.o. .d.e.s.p.r
>  data: 00 6f 00 74 00 65 00 67 00 69 00 64 00 61 00 20   .o.t.e.g.i.d.a.
>  data: 00 6e 00 6f 00 20 00 6d 00 65 00 73 00 20 00 70   .n.o. .m.e.s. .p
>  data: 00 61 00 73 00 73 00 61 00 64 00 6f 00 20 00 65   .a.s.s.a.d.o. .e
>  data: 00 20 00 f1 00 20 00 76 00 69 00 20 00 6d 00 65   . ... .v.i. .m.e
>  data: 00 73 00 74 00 72 00 75 00 61 00 e7 00 61 00 6f   .s.t.r.u.a...a.o
>  data: 00 20 00 70 00 6f 72 71 20 65 72 61 20 61 6e 74   . .p.orq era ant
>  data: 65 73 20 64 65 20 76 65 72 20 6e 61 71 75 65 6c   es de ver naquel
>  data: 65 20 6d 65 73 20 6d 61 73 20 64 65 70 6f 69 73   e mes mas depois
>  data: 20 6e 6f 20 69 6e 69 63 69 6f 20 64 65 73 74 65    no inicio deste
>  data: 20 65 6d 20 63 75 72 73 6f 20 76 69 20 75 6d 61    em curso vi uma
>  data: 73 20 67 6f 74 61 73 20 64 65 20 73 61 6e 67 75   s gotas de sangu
>  data: 65 20 6d 61 73 20 6d 75 69 74 6f 20 63 61 72 72   e mas muito carr
>  data: 65 67 61 64 61 20 64 75 72 61 6e 74 65 20 64 6f   egada durante do
>  data: 69 73 20 64 69 61 73 2e 73 65 72 61 71 20 73 74   is dias.seraq st
>  data: 6f 75 20 67 72 61 76 69 64 61 3f                  ou gravida?
> Octet string dump ends.
>
> ------------------------------------------------------------------------
> Part 1 delivered:
>
> SMPP PDU 0x7f191880cb40 dump:
>  type_name: deliver_sm
>  command_id: 5 = 0x00000005
>  command_status: 0 = 0x00000000
>  sequence_number: 294621 = 0x00047edd
>  service_type: NULL
>  source_addr_ton: 1 = 0x00000001
>  source_addr_npi: 1 = 0x00000001
>  source_addr: "258844096933"
>  dest_addr_ton: 0 = 0x00000000
>  dest_addr_npi: 1 = 0x00000001
>  destination_addr: "92222"
>  esm_class: 64 = 0x00000040
>  protocol_id: 0 = 0x00000000
>  priority_flag: 0 = 0x00000000
>  schedule_delivery_time: NULL
>  validity_period: NULL
>  registered_delivery: 0 = 0x00000000
>  replace_if_present_flag: 0 = 0x00000000
>  data_coding: 8 = 0x00000008
>  sm_default_msg_id: 0 = 0x00000000
>  sm_length: 140 = 0x0000008c
>  short_message:
>   Octet string at 0x7f191880cd70:
>     len:  140
>     size: 141
>     immutable: 0
>     data: 05 00 03 0d 04 01 00 45 00 75 00 20 00 74 00 69   .......E.u. .t.i
>     data: 00 76 00 20 00 75 00 6d 00 61 00 20 00 72 00 65   .v. .u.m.a. .r.e
>     data: 00 6c 00 61 00 e7 00 61 00 6f 00 20 00 64 00 65   .l.a...a.o. .d.e
>     data: 00 73 00 70 00 72 00 6f 00 74 00 65 00 67 00 69   .s.p.r.o.t.e.g.i
>     data: 00 64 00 61 00 20 00 6e 00 6f 00 20 00 6d 00 65   .d.a. .n.o. .m.e
>     data: 00 73 00 20 00 70 00 61 00 73 00 73 00 61 00 64   .s. .p.a.s.s.a.d
>     data: 00 6f 00 20 00 65 00 20 00 f1 00 20 00 76 00 69   .o. .e. ... .v.i
>     data: 00 20 00 6d 00 65 00 73 00 74 00 72 00 75 00 61   . .m.e.s.t.r.u.a
>     data: 00 e7 00 61 00 6f 00 20 00 70 00 6f               ...a.o. .p.o
>   Octet string dump ends.
> SMPP PDU dump ends.
> SMPP[unicef_mz_vodacom]: UDH length read as 6
> Got part 1 [ref 13, total parts 4] of message from +258844096933. Dump follows:
>
> ------------------------------------------------------------------------
> Part 2 delivered:
>
> SMPP PDU 0x7f1918809240 dump:
>  type_name: deliver_sm
>  command_id: 5 = 0x00000005
>  command_status: 0 = 0x00000000
>  sequence_number: 294618 = 0x00047eda
>  service_type: NULL
>  source_addr_ton: 1 = 0x00000001
>  source_addr_npi: 1 = 0x00000001
>  source_addr: "258844096933"
>  dest_addr_ton: 0 = 0x00000000
>  dest_addr_npi: 1 = 0x00000001
>  destination_addr: "92222"
>  esm_class: 64 = 0x00000040
>  protocol_id: 0 = 0x00000000
>  priority_flag: 0 = 0x00000000
>  schedule_delivery_time: NULL
>  validity_period: NULL
>  registered_delivery: 0 = 0x00000000
>  replace_if_present_flag: 0 = 0x00000000
>  data_coding: 0 = 0x00000000
>  sm_default_msg_id: 0 = 0x00000000
>  sm_length: 73 = 0x00000049
>  short_message:
>   Octet string at 0x7f1918811130:
>     len:  73
>     size: 74
>     immutable: 0
>     data: 05 00 03 0d 04 02 72 71 20 65 72 61 20 61 6e 74   ......rq era ant
>     data: 65 73 20 64 65 20 76 65 72 20 6e 61 71 75 65 6c   es de ver naquel
>     data: 65 20 6d 65 73 20 6d 61 73 20 64 65 70 6f 69 73   e mes mas depois
>     data: 20 6e 6f 20 69 6e 69 63 69 6f 20 64 65 73 74 65    no inicio deste
>     data: 20 65 6d 20 63 75 72 73 6f                         em curso
>   Octet string dump ends.
> SMPP PDU dump ends.
> SMPP[unicef_mz_vodacom]: UDH length read as 6
> Got part 2 [ref 13, total parts 4] of message from +258844096933. Dump follows:
>
> ------------------------------------------------------------------------
> Part 3 delivered:
>
> SMPP PDU 0x7f1918809240 dump:
>  type_name: deliver_sm
>  command_id: 5 = 0x00000005
>  command_status: 0 = 0x00000000
>  sequence_number: 294612 = 0x00047ed4
>  service_type: NULL
>  source_addr_ton: 1 = 0x00000001
>  source_addr_npi: 1 = 0x00000001
>  source_addr: "258844096933"
>  dest_addr_ton: 0 = 0x00000000
>  dest_addr_npi: 1 = 0x00000001
>  destination_addr: "92222"
>  esm_class: 64 = 0x00000040
>  protocol_id: 0 = 0x00000000
>  priority_flag: 0 = 0x00000000
>  schedule_delivery_time: NULL
>  validity_period: NULL
>  registered_delivery: 0 = 0x00000000
>  replace_if_present_flag: 0 = 0x00000000
>  data_coding: 0 = 0x00000000
>  sm_default_msg_id: 0 = 0x00000000
>  sm_length: 73 = 0x00000049
>  short_message:
>   Octet string at 0x7f191880be80:
>     len:  73
>     size: 74
>     immutable: 0
>     data: 05 00 03 0d 04 03 20 76 69 20 75 6d 61 73 20 67   ...... vi umas g
>     data: 6f 74 61 73 20 64 65 20 73 61 6e 67 75 65 20 6d   otas de sangue m
>     data: 61 73 20 6d 75 69 74 6f 20 63 61 72 72 65 67 61   as muito carrega
>     data: 64 61 20 64 75 72 61 6e 74 65 20 64 6f 69 73 20   da durante dois
>     data: 64 69 61 73 2e 73 65 72 61                        dias.sera
>   Octet string dump ends.
> SMPP PDU dump ends.
> SMPP[unicef_mz_vodacom]: UDH length read as 6
> Got part 3 [ref 13, total parts 4] of message from +258844096933. Dump follows:
>
> ------------------------------------------------------------------------
> Part 4 delivered:
>
> SMPP PDU 0x7f1918809240 dump:
>  type_name: deliver_sm
>  command_id: 5 = 0x00000005
>  command_status: 0 = 0x00000000
>  sequence_number: 294608 = 0x00047ed0
>  service_type: NULL
>  source_addr_ton: 1 = 0x00000001
>  source_addr_npi: 1 = 0x00000001
>  source_addr: "258844096933"
>  dest_addr_ton: 0 = 0x00000000
>  dest_addr_npi: 1 = 0x00000001
>  destination_addr: "92222"
>  esm_class: 64 = 0x00000040
>  protocol_id: 0 = 0x00000000
>  priority_flag: 0 = 0x00000000
>  schedule_delivery_time: NULL
>  validity_period: NULL
>  registered_delivery: 0 = 0x00000000
>  replace_if_present_flag: 0 = 0x00000000
>  data_coding: 0 = 0x00000000
>  sm_default_msg_id: 0 = 0x00000000
>  sm_length: 21 = 0x00000015
>  short_message:
>   Octet string at 0x7f1918800d30:
>     len:  21
>     size: 22
>     immutable: 0
>     data: 05 00 03 0d 04 04 71 20 73 74 6f 75 20 67 72 61   ......q stou gra
>     data: 76 69 64 61 3f                                    vida?
>   Octet string dump ends.
> SMPP PDU dump ends.
> SMPP[unicef_mz_vodacom]: UDH length read as 6
> Got part 4 [ref 13, total parts 4] of message from +258844096933. Dump follows:
>